[ZEPPELIN-804] Refactoring registration mechanism on Interpreters - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: In Progress
Priority: Major
Resolution: Unresolved
Affects Version/s: 0.5.6
Fix Version/s: None
Component/s: Interpreters
Labels:
None

Description

This issue is merged by https://github.com/apache/incubator-zeppelin/pull/835 and I'll make a several sub-tasks for other interpreters with beginner tags

I also found out some tests have used old way to register interpreters. I'll make sub-tasks for dealing with these.

Current registration mechanism has some problem the below:

Class.forName method works only if ClassLoader has all of jar or classes of imported libraries within a class.
Redundant step occurs for loading classes by separate ClassLoader. Because Zeppelin doesn't use a threaded Interpreter any more, We don't have to keep cleanCl any longer.

Proposal for new registration mechanism in interpreter/{interperter_group|interpareter_name}/conf/interpreter-setting.json. See example below:

[
  {
    "group": "spark",
    "name": "spark",
    "className": "org.apache.zeppelin.spark.SparkInterpreter"
    "properties": {
      "spark.app.name": { 
        "envName": "SPARK_APP_NAME",
        "propertyName": "spark.app.name",
        "default": "Zeppelin"
      },
      "master": {
        "envName": "MASTER",
        "propertyName": "spark.master",
        "default": "local[*]"
      },...
  },
  {
    "group": "spark",
    "name": "sql",
    "className": "org.apache.zeppelin.spark.SparkSqlInterpreter"
    ...
  }
]