Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-804

Refactoring registration mechanism on Interpreters

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.5.6
    • Fix Version/s: None
    • Component/s: Interpreters
    • Labels:
      None

      Description

      This issue is merged by https://github.com/apache/incubator-zeppelin/pull/835 and I'll make a several sub-tasks for other interpreters with beginner tags

      I also found out some tests have used old way to register interpreters. I'll make sub-tasks for dealing with these.


      Current registration mechanism has some problem the below:

      • Class.forName method works only if ClassLoader has all of jar or classes of imported libraries within a class.
      • Redundant step occurs for loading classes by separate ClassLoader. Because Zeppelin doesn't use a threaded Interpreter any more, We don't have to keep cleanCl any longer.

      Proposal for new registration mechanism in interpreter/{interperter_group|interpareter_name}/conf/interpreter-setting.json. See example below:

      [
        {
          "group": "spark",
          "name": "spark",
          "className": "org.apache.zeppelin.spark.SparkInterpreter"
          "properties": {
            "spark.app.name": { 
              "envName": "SPARK_APP_NAME",
              "propertyName": "spark.app.name",
              "default": "Zeppelin"
            },
            "master": {
              "envName": "MASTER",
              "propertyName": "spark.master",
              "default": "local[*]"
            },...
        },
        {
          "group": "spark",
          "name": "sql",
          "className": "org.apache.zeppelin.spark.SparkSqlInterpreter"
          ...
        }
      ]
      

        Attachments

        There are no Sub-Tasks for this issue.

          Activity

            People

            • Assignee:
              jongyoul Jongyoul Lee
              Reporter:
              jongyoul Jongyoul Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated: