Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
0.6.2
-
None
-
None
Description
For now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well.
Attachments
Issue Links
- blocks
-
ZEPPELIN-1596 Create ZeppelinContext for flink
- Closed
- is required by
-
ZEPPELIN-1967 Passing Z variables to Shell and SQL Interpreters
- Resolved
- links to
Activity
GitHub user zjffdu reopened a pull request:
https://github.com/apache/zeppelin/pull/1574
ZEPPELIN-1595. Make ZeppelinContext extensible
-
-
- What is this PR for?
For now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well.
- What is this PR for?
-
-
-
- What type of PR is it?
[Feature | Refactoring]
- What type of PR is it?
-
-
-
- Todos
-
- [ ] - Task
-
-
- What is the Jira issue?
-
-
-
- How should this be tested?
Outline the steps to test the PR here.
- How should this be tested?
-
-
-
- Screenshots (if appropriate)
-
-
-
- Questions:
-
- Does the licenses files need update? No
- Is there breaking changes for older versions? No
- Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1595
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1574.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1574
commit 95ca5e51a458cbb930a794e108eac85c73525d76
Author: Jeff Zhang <zjffdu@apache.org>
Date: 2016-10-31T07:33:51Z
ZEPPELIN-1595. Make ZeppelinContext extensible
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/1574
@Leemoonsoo @jongyoul @felixcheung Please help review.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1574
In my mind the longer term goal of Zeppelin Context is to support communication between interpreter groups. Would we be going the opposite direction somewhat by having interpreter/interpreter group specific Zeppelin Context, and so they are not longer shared?
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/1574
Do you mean ZeppelinContext would support communication between interpreter groups across JVM ? Currently ZeppelinContext only support spark interpreter, I don't see any case that it support communication between interpreter group. Maybe I miss something.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1574
To clarify, it doesn't right now, but I recall requests for that capability (which would be very useful for some of the more restricting interpreters to pass data around)
I will let others to comment?
GitHub user zjffdu reopened a pull request:
https://github.com/apache/zeppelin/pull/1574
ZEPPELIN-1595. Make ZeppelinContext extensible
-
-
- What is this PR for?
For now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well. For now, user need to implement the following 3 methods to extend ZeppelinContext
- What is this PR for?
-
- public List<Class> getSupportedClasses()
- public abstract Map<String, String> getInterpreterClassMap();
- protected abstract String showData(Object obj);
-
-
- What type of PR is it?
[Feature | Refactoring]
- What type of PR is it?
-
-
-
- Todos
-
- [ ] - Task
-
-
- What is the Jira issue?
-
-
-
- How should this be tested?
Outline the steps to test the PR here.
- How should this be tested?
-
-
-
- Screenshots (if appropriate)
-
-
-
- Questions:
-
- Does the licenses files need update? No
- Is there breaking changes for older versions? No
- Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1595
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1574.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1574
commit d05a2cd4f6a1b48ae0386b77e6b5c84b6c9eb619
Author: Jeff Zhang <zjffdu@apache.org>
Date: 2016-10-31T07:33:51Z
ZEPPELIN-1595. Make ZeppelinContext extensible
Github user wesm commented on the issue:
https://github.com/apache/zeppelin/pull/1574
In trying to investigate why the Apache Travis CI build queue is extremely backlogged, I found many build requests from this particular PR in https://travis-ci.org/apache/zeppelin/pull_requests
Since this project's build matrix has 11 entries, this can pretty easily jam up the build queue for other projects. I created https://issues.apache.org/jira/browse/INFRA-13469 in an attempt to mitigate this problem in the future
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1574
thanks @wesm for investigating this and following up. it's odd since this PR is 3 months old? though I'll add we should streamline the test matrix a bit (as we have from time to time)
Github user wesm commented on the issue:
https://github.com/apache/zeppelin/pull/1574
@felixcheung it looks like these builds were triggered by a merge on a different branch. see for example https://github.com/apache/zeppelin/commit/e259feacf78770b5dc412c0642216acdce17008f
in looking through your build history, it seems there is quite a bit of merging going on, and you may have been inadvertently triggering a lot of builds because of a quirk in Travis CI and GitHub's hooks: https://github.com/apache/zeppelin/commit/c4cf1abbf06ddf4447dacc886060ea4c1238464e
I would recommend advising the Zeppelin contributors to avoid using `git merge` altogether because of this and prefer instead to rebase or cherry-pick.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/1574
Thanks @wesm , I updated this PR yesterday, but didn't use `git merge`. I use `git rebase` and `git commit --amend`, does these also the issue as well ?
Github user wesm commented on the issue:
https://github.com/apache/zeppelin/pull/1574
Whatever you did generated a merge commit. The proof is right there – a rebate or amend would not have two parents
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/1574
@Leemoonsoo @felixcheung I update this PR, please help help review that. My purpose is to make ZeppelinContext extensible is in 2 perspective.
- All the interpreter can has its own ZeppelinContext. e.g. My next PR would to implement FlinkZeppelinContext which would display flink DataSet/Table data in zeppelin's table format
- Interpreters can also add customized features for its own ZeppelinContext. e.g. SparkZeppelinContext can add one extention point to allow user to visualize mllib model in zeppelin.
Communication between interpreter group mentioned by @felixcheung above is not implemented in this PR as I think this would bring much large change and need more sophisticated design. But we could put it in `BaseZeppelinContext` in future when we want to implement it.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/1574
@felixcheung @Leemoonsoo Would you mind to review this ?
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/1574
Great work. Looks good to me!
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/1574
Thanks for review, will merge if no more comments.
GitHub user zjffdu opened a pull request:
https://github.com/apache/zeppelin/pull/1574
ZEPPELIN-1595. Make ZeppelinContext extensibleFor now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well.
[Feature | Refactoring]
Outline the steps to test the PR here.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zjffdu/zeppelin
ZEPPELIN-1595Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1574.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1574
commit 0b6f7fefd275e1e3362e15fa85aed37b2f9c2007
Author: Jeff Zhang <zjffdu@apache.org>
Date: 2016-10-31T07:33:51Z
ZEPPELIN-1595. Make ZeppelinContext extensible