Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-5293

Enable Spark user applications to use different versions of Akka

    Details

    • Type: Umbrella
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: Spark Core
    • Labels:
      None
    • Target Version/s:

      Description

      A lot of Spark user applications are using (or want to use) Akka. Akka as a whole can contribute great architectural simplicity and uniformity. However, because Spark depends on Akka, it is not possible for users to rely on different versions, and we have received many requests in the past asking for help about this specific issue. For example, Spark Streaming might be used as the receiver of Akka messages - but our dependency on Akka requires the upstream Akka actors to also use the identical version of Akka.

      Since our usage of Akka is limited (mainly for RPC and single-threaded event loop), we can replace it with alternative RPC implementations and a common event loop in Spark.

        Issue Links

        There are no Sub-Tasks for this issue.

          Activity

          Hide
          CodingCat Nan Zhu added a comment -

          shall we make this JIRA as an umbrella task, so that the other JIRAs like SPARK-5214 can be associated to here?

          Show
          CodingCat Nan Zhu added a comment - shall we make this JIRA as an umbrella task, so that the other JIRAs like SPARK-5214 can be associated to here?
          Hide
          vanzin Marcelo Vanzin added a comment -

          I converted this bug to an umbrella so that it's easier to track sub-tasks. Hope that's ok for everybody.

          Show
          vanzin Marcelo Vanzin added a comment - I converted this bug to an umbrella so that it's easier to track sub-tasks. Hope that's ok for everybody.

            People

            • Assignee:
              zsxwing Shixiong Zhu
              Reporter:
              rxin Reynold Xin
            • Votes:
              9 Vote for this issue
              Watchers:
              37 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development