Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
User writes code and Zeppelin run and display the result on the notebook.
If user code can be packaged and reused, that would be really useful.
The idea is simple, run packaged user code on interpreter process, just like an user code on the notebook, and display result on notebook just like an paragraph result.
I'd like to call packaged user code as Application.
How user runs Application?
Zeppelin have list of applications in 'Registry'. Depends on resources available in paragraph, Zeppelin suggest possible applications as a icon button, just like Zeppelin suggest possible visualization for table result.
When user click the icon button, then an application is downloaded from 'Repository' and run on the interpreter process.
How application artifact looks like?
Application is downloadable / runnable artifact that contains user code that interact with Zeppelin provided API.
I think Jar format would be simplest way to go without inventing new package format. And we can leverage maven repository for package distribution.
What is Registry?
Each application is described by package metadata. package metadata contains package name, icon, description, artifact, version, required resources, license, etc.
Registry is collection of package metadata. Zeppelin community can maintain one central registry.
How application displays output?
Each paragraph has output message, angular objects, dynamic forms. Single paragraph will have multiple applications and each of them has their own output message, angular objects just like an paragraph output.
What is resources?
resource is object available in ResourcePool. It can be paragraph result, it can be something exposed from interpreter (e.g. SparkContext).
Package metadata describes required resource for the package. Zeppelin will scan resource available in the resourcepool and suggest possible package for the paragraph.
What this issue trying to do?
Minimal implementation of 'Application runtime'.
Which includes application download and loader, application selection icon, displaying output and examples.
Possible future work
This application framework can be extended to use built-in pivot function to transform input data. Then it becomes one possible implementation of pluggable visualization.
