Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: None
    • Labels:
      None

      Description

      User writes code and Zeppelin run and display the result on the notebook.
      If user code can be packaged and reused, that would be really useful.

      The idea is simple, run packaged user code on interpreter process, just like an user code on the notebook, and display result on notebook just like an paragraph result.

      I'd like to call packaged user code as Application.

      How user runs Application?

      Zeppelin have list of applications in 'Registry'. Depends on resources available in paragraph, Zeppelin suggest possible applications as a icon button, just like Zeppelin suggest possible visualization for table result.

      When user click the icon button, then an application is downloaded from 'Repository' and run on the interpreter process.

      How application artifact looks like?

      Application is downloadable / runnable artifact that contains user code that interact with Zeppelin provided API.
      I think Jar format would be simplest way to go without inventing new package format. And we can leverage maven repository for package distribution.

      What is Registry?

      Each application is described by package metadata. package metadata contains package name, icon, description, artifact, version, required resources, license, etc.
      Registry is collection of package metadata. Zeppelin community can maintain one central registry.

      How application displays output?

      Each paragraph has output message, angular objects, dynamic forms. Single paragraph will have multiple applications and each of them has their own output message, angular objects just like an paragraph output.

      What is resources?

      resource is object available in ResourcePool. It can be paragraph result, it can be something exposed from interpreter (e.g. SparkContext).
      Package metadata describes required resource for the package. Zeppelin will scan resource available in the resourcepool and suggest possible package for the paragraph.

      What this issue trying to do?

      Minimal implementation of 'Application runtime'.
      Which includes application download and loader, application selection icon, displaying output and examples.

      Possible future work

      This application framework can be extended to use built-in pivot function to transform input data. Then it becomes one possible implementation of pluggable visualization.

        Issue Links

          Activity

          Hide
          eric@apache.org Eric Charles added a comment -

          How does this relate (or not) to the Helium proposal?
          https://cwiki.apache.org/confluence/display/ZEPPELIN/Helium+proposal

          Show
          eric@apache.org Eric Charles added a comment - How does this relate (or not) to the Helium proposal? https://cwiki.apache.org/confluence/display/ZEPPELIN/Helium+proposal
          Hide
          moon Lee moon soo added a comment -

          Thanks for the interest.
          It's basically the same. with small improvements. for example, in helium proposal, proof of concept implementation requires new paragraph, and application runs in new paragraph against resource of previous paragraph. This approach also required user run %helium interpreter every time. This issue, trying to run applications in the same paragraph, to make user experience more simple.

          Show
          moon Lee moon soo added a comment - Thanks for the interest. It's basically the same. with small improvements. for example, in helium proposal, proof of concept implementation requires new paragraph, and application runs in new paragraph against resource of previous paragraph. This approach also required user run %helium interpreter every time. This issue, trying to run applications in the same paragraph, to make user experience more simple.
          Hide
          eric@apache.org Eric Charles added a comment -

          Thx Moon for clarification. Should the spec listed in this JIRA added to the Helium spec?
          What are your plan(ning) to implement this? I was thinking to contributed in a way or the other: testing implementation, reviewing code, contributing requirements and/or code...
          For example, are your current helium branch and current 3 sample projects the place to further work this?

          Show
          eric@apache.org Eric Charles added a comment - Thx Moon for clarification. Should the spec listed in this JIRA added to the Helium spec? What are your plan(ning) to implement this? I was thinking to contributed in a way or the other: testing implementation, reviewing code, contributing requirements and/or code... For example, are your current helium branch and current 3 sample projects the place to further work this?
          Hide
          moon Lee moon soo added a comment -

          Right, let me update Helium proposal. I'm working on implementation of this issue. If you can help review and test, that would be appreciated. Also there're many unfinished/undefined works in ZEPPELIN-533. Any contribution on them would be appreciated, too.

          Show
          moon Lee moon soo added a comment - Right, let me update Helium proposal. I'm working on implementation of this issue. If you can help review and test, that would be appreciated. Also there're many unfinished/undefined works in ZEPPELIN-533 . Any contribution on them would be appreciated, too.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user Leemoonsoo opened a pull request:

          https://github.com/apache/incubator-zeppelin/pull/836

          [WIP] ZEPPELIN-732 Helium Application

              1. What is this PR for?
                This PR implements pluggable Application Framework.

          Here's summary of the change.

          *Applicaiton*

          Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`.

          It looks like

          ```java
          public abstract class Application {

          /**

          • When application loaded
          • @params context Application context
            */
            public Application(ApplicationContext context) { this.context = context; }

          /**

          • This method can be invoked multiple times before unload(),
          • Invoked just after application loaded or when paragraph output is updated
          • @param args Required resources from paragraph
            */
            public abstract void run(ResourceSet args)
            throws ApplicationException, IOException;

          /**

          • this method is invoked just before application is removed
            */
            public abstract void unload() throws ApplicationException;
            }
            ```

          *HeliumPackage*

          `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package.
          Package informations includes

          ```

          { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon }

          ```

          `resources` field defines what kind of data this application requires, from ResourcePool.
          "resourceName" and ":className" works in the array.
          inner array combines each resourceName, :className with 'AND'
          outer array combines inner array with 'OR'.

          For example,

          ```
          resources : [
          [ "name1", ":java.util.Date"],
          [ "name1", "name2"]
          ]
          ```

          Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2".

          Once resources are found, they'll be passed to `Application.run()` method

          *Package Registry*

          `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages.
          Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry.

          *Development mode*
          `org.apache.zeppelin.interpreter.dev.*` package is provided for development support.
          Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output.

          *Application Suggestion*
          Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed.

          ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png)

          User can see and select available application for this paragraph.
          ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png)

          *Application selection*
          After application is loaded, application icon will be displayed. So user can select and switch between apps.

          ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png)

          if built-in visualization is available, application icon will be displayed next to built-in visualizations
          ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png)

          *Experimental*

          While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while.

              1. What type of PR is it?
                Feature
              1. Todos
          • [x] - Helium Application framework
          • [x] - App launcher
          • [x] - App display and selectors
          • [x] - Package registry
          • [x] - Development mode
          • [ ] - Make CI green
          • [ ] - Improve comment in source codes
          • [ ] - Documentation
          • [ ] - Examples
          • [ ] - Mark experimental
          • [ ] - Review
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-732
              1. How should this be tested?
                Will be updated with examples
              1. Screenshots (if appropriate)
                ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif)
              1. Questions:
          • Does the licenses files need update? no
          • Is there breaking changes for older versions? no
          • Does this needs documentation? yes

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/Leemoonsoo/incubator-zeppelin ZEPPELIN-732-up

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/incubator-zeppelin/pull/836.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #836


          commit 7424af247b08afabbd9d080a288a88a7de4b7a1c
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-03-05T06:08:19Z

          Remove resource on note / paragraph removal

          commit 568ee541db31133ccd8f2d99d16ede60cb9433c1
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-03-07T02:55:17Z

          ApplicationLoader

          commit b891b98ee5a3bd9dde8c044b32c0dc09074cdcb0
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-03-13T23:13:49Z

          HeliumRegistry

          commit b239f1b96b01264b4f92aef0d5d05d5173794c3b
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-03-19T15:25:30Z

          Helium application factory

          commit 9f5c493e414e7c240abb08f2fa8600761514d63f
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-01T18:41:57Z

          Application output

          commit 4eaeea720561962766f25438b19d7a334a8cb0bc
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-02T20:18:54Z

          sync -> async api

          commit 7aeb64addf55e0712aa270e131f4af39d06f470c
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-03T03:14:23Z

          Unload app on paragraph / note removal as well as interpreter unbind

          commit 134bbe640ae2636bb847d9844b28f87409027d08
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-06T01:32:47Z

          Change HeliumRegistry constructor argument type

          commit 94b490d76c7875a8d2f1a30232a762ec374664b4
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-06T01:35:37Z

          initial rest api impl

          commit bd0f467d23aecd32adf0152b939d8ed8eea6f56f
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-09T00:38:24Z

          Style

          commit 6223cd44d9fb3b7190c2cf7e817a031e38b5eb6f
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-12T01:10:58Z

          App output display

          commit 16f68871daaf98b4d3d4ca759b3f750e597bf5a4
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-12T04:53:34Z

          Angular object update for helium app

          commit 412480a2d222e31c7bd74198c5a4249dfea7ba8b
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-12T05:33:45Z

          Fix style

          commit be3a1fa7b3da908051605945926790e1342ffd0a
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-12T05:52:15Z

          Add license header

          commit 5503f9c100bfde93f5c3eb2f12bf88e6e18ab65b
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-12T12:13:16Z

          Improved

          commit b47ca744b989d8644e7cafe4ec3f2d336b890f92
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-12T22:46:45Z

          Fix tests

          commit 98f3872c6a27e8699717b9fc20b353b8fe9a1d2d
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-13T00:13:49Z

          Managed interpreter process and Running interpreter process

          commit 024d7fc2c52c54a6b5925598330d6326194fa8b4
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-13T08:37:40Z

          Dev mode

          commit 71f814dce41113f71fdcfff4c7d468604cd35f4b
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-13T12:29:34Z

          Better way to find resource dir for InterpreterOutput watcher

          commit f2ab95dcd37e2968c9c7c75f595d32bc2ef56dee
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-13T22:22:45Z

          Prevent unnecessary output update

          commit b4ff52fbcf7f0d7dbeac3d56f8ab4f4c283cdeba
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-14T00:05:54Z

          Put last value of scala repl into resource pool

          commit ec2fdea60bcd7e10438e2cc8d888d6995a2151f8
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-14T00:26:33Z

          Match classname correctly

          commit 0e4d81c841791ea4a4bd580478e57051d61a8b10
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-14T00:26:51Z

          Remove unnecessary log

          commit fade3c1f0f04603d28ce594a672f35e9558b7499
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-14T02:48:10Z

          Handle output update in angular mode

          commit 03be3a12056ae96e47de128d32a00860ac0786a3
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-14T04:24:43Z

          Pass required resource to run() method

          commit c30f53c7bb77c84c9fc07bf72674811bae04a064
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-14T05:03:35Z

          null check

          commit 864bea14025465d6f98bdc854ccf2c007f57b0d1
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-04-14T05:23:19Z

          Merge branch 'master' into ZEPPELIN-732-up


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user Leemoonsoo opened a pull request: https://github.com/apache/incubator-zeppelin/pull/836 [WIP] ZEPPELIN-732 Helium Application What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. * Applicaiton * Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** When application loaded @params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** This method can be invoked multiple times before unload(), Invoked just after application loaded or when paragraph output is updated @param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` * HeliumPackage * `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method * Package Registry * `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. * Development mode * `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. * Application Suggestion * Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ! [image] ( https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png ) User can see and select available application for this paragraph. ! [image] ( https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png ) * Application selection * After application is loaded, application icon will be displayed. So user can select and switch between apps. ! [image] ( https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png ) if built-in visualization is available, application icon will be displayed next to built-in visualizations ! [image] ( https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png ) * Experimental * While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. What type of PR is it? Feature Todos [x] - Helium Application framework [x] - App launcher [x] - App display and selectors [x] - Package registry [x] - Development mode [ ] - Make CI green [ ] - Improve comment in source codes [ ] - Documentation [ ] - Examples [ ] - Mark experimental [ ] - Review What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 How should this be tested? Will be updated with examples Screenshots (if appropriate) ! [helium] ( https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif ) Questions: Does the licenses files need update? no Is there breaking changes for older versions? no Does this needs documentation? yes You can merge this pull request into a Git repository by running: $ git pull https://github.com/Leemoonsoo/incubator-zeppelin ZEPPELIN-732 -up Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-zeppelin/pull/836.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #836 commit 7424af247b08afabbd9d080a288a88a7de4b7a1c Author: Lee moon soo <moon@apache.org> Date: 2016-03-05T06:08:19Z Remove resource on note / paragraph removal commit 568ee541db31133ccd8f2d99d16ede60cb9433c1 Author: Lee moon soo <moon@apache.org> Date: 2016-03-07T02:55:17Z ApplicationLoader commit b891b98ee5a3bd9dde8c044b32c0dc09074cdcb0 Author: Lee moon soo <moon@apache.org> Date: 2016-03-13T23:13:49Z HeliumRegistry commit b239f1b96b01264b4f92aef0d5d05d5173794c3b Author: Lee moon soo <moon@apache.org> Date: 2016-03-19T15:25:30Z Helium application factory commit 9f5c493e414e7c240abb08f2fa8600761514d63f Author: Lee moon soo <moon@apache.org> Date: 2016-04-01T18:41:57Z Application output commit 4eaeea720561962766f25438b19d7a334a8cb0bc Author: Lee moon soo <moon@apache.org> Date: 2016-04-02T20:18:54Z sync -> async api commit 7aeb64addf55e0712aa270e131f4af39d06f470c Author: Lee moon soo <moon@apache.org> Date: 2016-04-03T03:14:23Z Unload app on paragraph / note removal as well as interpreter unbind commit 134bbe640ae2636bb847d9844b28f87409027d08 Author: Lee moon soo <moon@apache.org> Date: 2016-04-06T01:32:47Z Change HeliumRegistry constructor argument type commit 94b490d76c7875a8d2f1a30232a762ec374664b4 Author: Lee moon soo <moon@apache.org> Date: 2016-04-06T01:35:37Z initial rest api impl commit bd0f467d23aecd32adf0152b939d8ed8eea6f56f Author: Lee moon soo <moon@apache.org> Date: 2016-04-09T00:38:24Z Style commit 6223cd44d9fb3b7190c2cf7e817a031e38b5eb6f Author: Lee moon soo <moon@apache.org> Date: 2016-04-12T01:10:58Z App output display commit 16f68871daaf98b4d3d4ca759b3f750e597bf5a4 Author: Lee moon soo <moon@apache.org> Date: 2016-04-12T04:53:34Z Angular object update for helium app commit 412480a2d222e31c7bd74198c5a4249dfea7ba8b Author: Lee moon soo <moon@apache.org> Date: 2016-04-12T05:33:45Z Fix style commit be3a1fa7b3da908051605945926790e1342ffd0a Author: Lee moon soo <moon@apache.org> Date: 2016-04-12T05:52:15Z Add license header commit 5503f9c100bfde93f5c3eb2f12bf88e6e18ab65b Author: Lee moon soo <moon@apache.org> Date: 2016-04-12T12:13:16Z Improved commit b47ca744b989d8644e7cafe4ec3f2d336b890f92 Author: Lee moon soo <moon@apache.org> Date: 2016-04-12T22:46:45Z Fix tests commit 98f3872c6a27e8699717b9fc20b353b8fe9a1d2d Author: Lee moon soo <moon@apache.org> Date: 2016-04-13T00:13:49Z Managed interpreter process and Running interpreter process commit 024d7fc2c52c54a6b5925598330d6326194fa8b4 Author: Lee moon soo <moon@apache.org> Date: 2016-04-13T08:37:40Z Dev mode commit 71f814dce41113f71fdcfff4c7d468604cd35f4b Author: Lee moon soo <moon@apache.org> Date: 2016-04-13T12:29:34Z Better way to find resource dir for InterpreterOutput watcher commit f2ab95dcd37e2968c9c7c75f595d32bc2ef56dee Author: Lee moon soo <moon@apache.org> Date: 2016-04-13T22:22:45Z Prevent unnecessary output update commit b4ff52fbcf7f0d7dbeac3d56f8ab4f4c283cdeba Author: Lee moon soo <moon@apache.org> Date: 2016-04-14T00:05:54Z Put last value of scala repl into resource pool commit ec2fdea60bcd7e10438e2cc8d888d6995a2151f8 Author: Lee moon soo <moon@apache.org> Date: 2016-04-14T00:26:33Z Match classname correctly commit 0e4d81c841791ea4a4bd580478e57051d61a8b10 Author: Lee moon soo <moon@apache.org> Date: 2016-04-14T00:26:51Z Remove unnecessary log commit fade3c1f0f04603d28ce594a672f35e9558b7499 Author: Lee moon soo <moon@apache.org> Date: 2016-04-14T02:48:10Z Handle output update in angular mode commit 03be3a12056ae96e47de128d32a00860ac0786a3 Author: Lee moon soo <moon@apache.org> Date: 2016-04-14T04:24:43Z Pass required resource to run() method commit c30f53c7bb77c84c9fc07bf72674811bae04a064 Author: Lee moon soo <moon@apache.org> Date: 2016-04-14T05:03:35Z null check commit 864bea14025465d6f98bdc854ccf2c007f57b0d1 Author: Lee moon soo <moon@apache.org> Date: 2016-04-14T05:23:19Z Merge branch 'master' into ZEPPELIN-732 -up
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jongyoul commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/836#issuecomment-209955698

          @Leemoonsoo I've thought Helium is a kind of backend system. But as I saw your screenshot, it looks like a frontend system. Do you have any change? or does Helium support both of them?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jongyoul commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/836#issuecomment-209955698 @Leemoonsoo I've thought Helium is a kind of backend system. But as I saw your screenshot, it looks like a frontend system. Do you have any change? or does Helium support both of them?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Leemoonsoo commented on the pull request:

          https://github.com/apache/incubator-zeppelin/pull/836#issuecomment-210724384

          @jongyoul Thanks for the interest.

          Helium just runs jar library (which extends `Application`). Which may include some output and the output is displayed in Zeppelin notebook. And the output is always handled by Angular display system. So displayed output can interact with the code that runs on backend.

          Does this approach makes sense? what do you think?

          Show
          githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the pull request: https://github.com/apache/incubator-zeppelin/pull/836#issuecomment-210724384 @jongyoul Thanks for the interest. Helium just runs jar library (which extends `Application`). Which may include some output and the output is displayed in Zeppelin notebook. And the output is always handled by Angular display system. So displayed output can interact with the code that runs on backend. Does this approach makes sense? what do you think?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Leemoonsoo commented on the issue:

          https://github.com/apache/zeppelin/pull/836

          It's ready for review. This PR provides experimental Java API to add pluggable components.
          Which is still i think pretty low level.

          Adding high level api (e.g. ZEPPELIN-1003(https://issues.apache.org/jira/browse/ZEPPELIN-1003)) and providing API for other languages (e.g. scala, python, javascript, etc) will be the future work.

          Show
          githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the issue: https://github.com/apache/zeppelin/pull/836 It's ready for review. This PR provides experimental Java API to add pluggable components. Which is still i think pretty low level. Adding high level api (e.g. ZEPPELIN-1003 ( https://issues.apache.org/jira/browse/ZEPPELIN-1003 )) and providing API for other languages (e.g. scala, python, javascript, etc) will be the future work.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/836

          Great stuff!! It's going to take some time to review.

          @Leemoonsoo @minahlee Please help me understand, do you think this should be included in 0.6.0 release? As I do not see ZEPPELIN-732(https://issues.apache.org/jira/browse/ZEPPELIN-732) under ZEPPELIN-889(https://issues.apache.org/jira/browse/ZEPPELIN-889)

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/836 Great stuff!! It's going to take some time to review. @Leemoonsoo @minahlee Please help me understand, do you think this should be included in 0.6.0 release? As I do not see ZEPPELIN-732 ( https://issues.apache.org/jira/browse/ZEPPELIN-732 ) under ZEPPELIN-889 ( https://issues.apache.org/jira/browse/ZEPPELIN-889 )
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/836

          @khalidhuseynov Just FYI, this PR adds 4 new messages websocket protocol:

          ```
          APP_APPEND_OUTPUT, // [s-c] append output
          APP_UPDATE_OUTPUT, // [s-c] update (replace) output
          APP_LOAD, // [s-c] on app load
          APP_STATUS_CHANGE, // [s-c] on app status change
          ```

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/836 @khalidhuseynov Just FYI, this PR adds 4 new messages websocket protocol: ``` APP_APPEND_OUTPUT, // [s-c] append output APP_UPDATE_OUTPUT, // [s-c] update (replace) output APP_LOAD, // [s-c] on app load APP_STATUS_CHANGE, // [s-c] on app status change ```
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Leemoonsoo commented on the issue:

          https://github.com/apache/zeppelin/pull/836

          @bzz Pluggable visualization is one of [roadmap of 0.6 release](https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap), and i think this PR provides a way to enable it.

          So if we can merge it into 0.6, it'll be better. but this feature will not be blocker for 0.6.0 release.

          Show
          githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the issue: https://github.com/apache/zeppelin/pull/836 @bzz Pluggable visualization is one of [roadmap of 0.6 release] ( https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap ), and i think this PR provides a way to enable it. So if we can merge it into 0.6, it'll be better. but this feature will not be blocker for 0.6.0 release.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bzz commented on the issue:

          https://github.com/apache/zeppelin/pull/836

          Looks great, do you plan to rebase on master as well?

          I think there was also @anish18sun using it under ZEPPELIN-684(https://issues.apache.org/jira/browse/ZEPPELIN-684) so \cc @anish18sun for feedback as well

          Show
          githubbot ASF GitHub Bot added a comment - Github user bzz commented on the issue: https://github.com/apache/zeppelin/pull/836 Looks great, do you plan to rebase on master as well? I think there was also @anish18sun using it under ZEPPELIN-684 ( https://issues.apache.org/jira/browse/ZEPPELIN-684 ) so \cc @anish18sun for feedback as well
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user anish18sun commented on the issue:

          https://github.com/apache/zeppelin/pull/836

          read all the code changes, I think the usage of print() and println() around context().out.write() in Application.java is really intuitive for users!

          Show
          githubbot ASF GitHub Bot added a comment - Github user anish18sun commented on the issue: https://github.com/apache/zeppelin/pull/836 read all the code changes, I think the usage of print() and println() around context().out.write() in Application.java is really intuitive for users!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Leemoonsoo commented on the issue:

          https://github.com/apache/zeppelin/pull/836

          Addressed comment from @bzz. @anish18sun Thanks for taking a look.

          Now CI is green and i'll keep this PR open in this weekend for further reviews. if there're no further discussions, then i'll merge it into the master.

          Show
          githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the issue: https://github.com/apache/zeppelin/pull/836 Addressed comment from @bzz. @anish18sun Thanks for taking a look. Now CI is green and i'll keep this PR open in this weekend for further reviews. if there're no further discussions, then i'll merge it into the master.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Leemoonsoo commented on the issue:

          https://github.com/apache/zeppelin/pull/836

          Thanks @AhyoungRyu for the review. Comments are addressed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the issue: https://github.com/apache/zeppelin/pull/836 Thanks @AhyoungRyu for the review. Comments are addressed.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user AhyoungRyu commented on the issue:

          https://github.com/apache/zeppelin/pull/836

          @Leemoonsoo Thanks. Awesome work indeed!

          Show
          githubbot ASF GitHub Bot added a comment - Github user AhyoungRyu commented on the issue: https://github.com/apache/zeppelin/pull/836 @Leemoonsoo Thanks. Awesome work indeed!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/zeppelin/pull/836

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/836
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user AhyoungRyu opened a pull request:

          https://github.com/apache/zeppelin/pull/1133

          [DOC]ZEPPELIN-732 Helium Application

              1. What is this PR for?
                After #836 and #1031 merged into master branch, I also applied TOC(table of contents) to newly added `writingzeppelinapplication.md`. And also added this docs link under `index.md`'s docs list.
              1. What type of PR is it?
                Documentation
              1. Todos
          • [x] - Apply TOC(table of contents) to `writingzeppelinapplication.md` and add this docs to `index.md`
              1. Questions:
          • Does the licenses files need update? no
          • Is there breaking changes for older versions? no
          • Does this needs documentation? no

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/AhyoungRyu/zeppelin docs/ZEPPELIN-732

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1133.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1133


          commit 7dc0b9547a9158747854d2a3fc3748ff7e15f213
          Author: AhyoungRyu <fbdkdud93@hanmail.net>
          Date: 2016-07-05T08:33:03Z

          Add writingzeppelinapplication.md to index.md

          commit e144e0d6ffdc578509ab35c6f03a4c35f21a51d3
          Author: AhyoungRyu <fbdkdud93@hanmail.net>
          Date: 2016-07-05T08:33:29Z

          Apply toc to writingzeppelinapplication.md

          commit 9fc746fc4b1981648f40c13126c3ab07893b6135
          Author: AhyoungRyu <fbdkdud93@hanmail.net>
          Date: 2016-07-05T08:36:32Z

          Fix a grammar error


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user AhyoungRyu opened a pull request: https://github.com/apache/zeppelin/pull/1133 [DOC] ZEPPELIN-732 Helium Application What is this PR for? After #836 and #1031 merged into master branch, I also applied TOC(table of contents) to newly added `writingzeppelinapplication.md`. And also added this docs link under `index.md`'s docs list. What type of PR is it? Documentation Todos [x] - Apply TOC(table of contents) to `writingzeppelinapplication.md` and add this docs to `index.md` Questions: Does the licenses files need update? no Is there breaking changes for older versions? no Does this needs documentation? no You can merge this pull request into a Git repository by running: $ git pull https://github.com/AhyoungRyu/zeppelin docs/ ZEPPELIN-732 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1133.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1133 commit 7dc0b9547a9158747854d2a3fc3748ff7e15f213 Author: AhyoungRyu <fbdkdud93@hanmail.net> Date: 2016-07-05T08:33:03Z Add writingzeppelinapplication.md to index.md commit e144e0d6ffdc578509ab35c6f03a4c35f21a51d3 Author: AhyoungRyu <fbdkdud93@hanmail.net> Date: 2016-07-05T08:33:29Z Apply toc to writingzeppelinapplication.md commit 9fc746fc4b1981648f40c13126c3ab07893b6135 Author: AhyoungRyu <fbdkdud93@hanmail.net> Date: 2016-07-05T08:36:32Z Fix a grammar error
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Leemoonsoo commented on the issue:

          https://github.com/apache/zeppelin/pull/1133

          Thank you @AhyoungRyu for taking care of it.
          Looks good to me!

          Show
          githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the issue: https://github.com/apache/zeppelin/pull/1133 Thank you @AhyoungRyu for taking care of it. Looks good to me!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Leemoonsoo commented on the issue:

          https://github.com/apache/zeppelin/pull/1133

          Merge it into master if there're no more discussions

          Show
          githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the issue: https://github.com/apache/zeppelin/pull/1133 Merge it into master if there're no more discussions
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/zeppelin/pull/1133

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/1133
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user Leemoonsoo opened a pull request:

          https://github.com/apache/zeppelin/pull/1150

          [HOTFIX] Fix NullPointerException at org.apache.zeppelin.notebook.Paragraph.getRepl(Paragraph.java:204)

              1. What is this PR for?
                When notebook is being loaded, following error is raised after #836

          ```
          Caused by: java.lang.NullPointerException
          at org.apache.zeppelin.notebook.Paragraph.getRepl(Paragraph.java:204)
          at org.apache.zeppelin.notebook.Paragraph.getCurrentRepl(Paragraph.java:208)
          at org.apache.zeppelin.helium.Helium.suggestApp(Helium.java:138)
          at org.apache.zeppelin.rest.HeliumRestApi.suggest(HeliumRestApi.java:83)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:606)
          at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
          at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
          ... 50 more
          ```

              1. What type of PR is it?
                Hotfix
              1. Todos
          • [x] - Fix
              1. What is the Jira issue?
                ZEPPELIN-732, ZEPPELIN-1075
              1. How should this be tested?
                Run Zeppelin and load any notebook. And see if there're unexpected exception raised.
              1. Questions:
          • Does the licenses files need update? no
          • Is there breaking changes for older versions? no
          • Does this needs documentation? no

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/Leemoonsoo/zeppelin fix_npe

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1150.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1150


          commit 3e8697b675bc1682ab6348b810a2f19fb827cee5
          Author: Lee moon soo <moon@apache.org>
          Date: 2016-07-07T23:51:39Z

          Set interpreter factory into paragraph right after loaded


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user Leemoonsoo opened a pull request: https://github.com/apache/zeppelin/pull/1150 [HOTFIX] Fix NullPointerException at org.apache.zeppelin.notebook.Paragraph.getRepl(Paragraph.java:204) What is this PR for? When notebook is being loaded, following error is raised after #836 ``` Caused by: java.lang.NullPointerException at org.apache.zeppelin.notebook.Paragraph.getRepl(Paragraph.java:204) at org.apache.zeppelin.notebook.Paragraph.getCurrentRepl(Paragraph.java:208) at org.apache.zeppelin.helium.Helium.suggestApp(Helium.java:138) at org.apache.zeppelin.rest.HeliumRestApi.suggest(HeliumRestApi.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180) at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96) ... 50 more ``` What type of PR is it? Hotfix Todos [x] - Fix What is the Jira issue? ZEPPELIN-732 , ZEPPELIN-1075 How should this be tested? Run Zeppelin and load any notebook. And see if there're unexpected exception raised. Questions: Does the licenses files need update? no Is there breaking changes for older versions? no Does this needs documentation? no You can merge this pull request into a Git repository by running: $ git pull https://github.com/Leemoonsoo/zeppelin fix_npe Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1150.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1150 commit 3e8697b675bc1682ab6348b810a2f19fb827cee5 Author: Lee moon soo <moon@apache.org> Date: 2016-07-07T23:51:39Z Set interpreter factory into paragraph right after loaded
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/zeppelin/pull/1150

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/1150

            People

            • Assignee:
              moon Lee moon soo
              Reporter:
              moon Lee moon soo
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development