Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10745

Separate configs between shuffle and RPC

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.6.0
    • Spark Core
    • None

    Description

      SPARK-6028 uses network module to implement RPC. However, there are some configurations named with `spark.shuffle` prefix in the network module. We should refactor them and make sure the user can control them in shuffle and RPC separately.

      Attachments

        Issue Links

          blocks

          Documentation - Documentation or Website SPARK-11822 Add docs for new Netty RPC configuration

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is blocked by

          Sub-task - The sub-task of the issue SPARK-6028 Provide an alternative RPC implementation based on the network transport module

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Pull request #9481 [Github] Pull Request #9481 (zsxwing)

          Activity

            People

              zsxwing Shixiong Zhu
              zsxwing Shixiong Zhu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: