Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-3111

Refactor SparkInterpreter

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.8.0
    • None
    • None

    Description

      The current spark interpreter is pretty complex. Because of 2 reasons:

      • It is written by java and it uses a lot of java reflection to call spark’s scala api.
      • It mix both scala 2.10 & scala 2.11.

      With more spark versions are released and more scala version supported (spark 2.3 will support scala 2.12). Zeppelin’s spark interpreter would become more and more complex and pretty hard to maintain. So I propose to refactor the spark interpreter.

      Attach the design doc. \cc moon felixcheung jongyoul

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. ZEPPELIN-2385 Release 0.8.0

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. ZEPPELIN-3807 Fix spark interpreter on Windows

          • Major - Major loss of function.
          • Open
          links to

          Web Link Design Doc

          Web Link GitHub Pull Request #2709

          Activity

            People

              zjffdu Jeff Zhang
              zjffdu Jeff Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: