Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6017 Provide transparent secure communication channel on Yarn
  3. SPARK-11563

Use RpcEnv to transfer generated classes in spark-shell

          Details

          • Type: Sub-task
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: None
          • Fix Version/s: 2.0.0
          • Component/s: Spark Shell
          • Labels:
            None

            Description

            Similar to SPARK-11140 (and building on top of it), spark-shell should transfer generated classes using the RpcEnv support for file transfers added in that change.

            That would mean easy security configuration and, as a bonus, lower overhead (and one less HTTP server running inside Spark).

              Attachments

                Issue Links

                is blocked by

                Sub-task - The sub-task of the issue SPARK-11140 Replace file server in driver with RPC-based alternative

                • Major - Major loss of function.
                • Resolved
                links to

                Pull request #9923 [Github] Pull Request #9923 (vanzin)

                  Activity

                    People

                    • Assignee:
                      vanzin Marcelo Vanzin
                      Reporter:
                      vanzin Marcelo Vanzin
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      2 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: