Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.6.2
    • 0.8.0
    • None
    • None

    Description

      For now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well.

      Attachments

        Issue Links

          Activity

            githubbot ASF GitHub Bot added a comment -

            GitHub user zjffdu opened a pull request:

            https://github.com/apache/zeppelin/pull/1574

            ZEPPELIN-1595. Make ZeppelinContext extensible

                1. What is this PR for?
                  For now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well.
                1. What type of PR is it?
                  [Feature | Refactoring]
                1. Todos
            • [ ] - Task
                1. What is the Jira issue?
                1. How should this be tested?
                  Outline the steps to test the PR here.
                1. Screenshots (if appropriate)
                1. Questions:
            • Does the licenses files need update? No
            • Is there breaking changes for older versions? No
            • Does this needs documentation? No

            You can merge this pull request into a Git repository by running:

            $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1595

            Alternatively you can review and apply these changes as the patch at:

            https://github.com/apache/zeppelin/pull/1574.patch

            To close this pull request, make a commit to your master/trunk branch
            with (at least) the following in the commit message:

            This closes #1574


            commit 0b6f7fefd275e1e3362e15fa85aed37b2f9c2007
            Author: Jeff Zhang <zjffdu@apache.org>
            Date: 2016-10-31T07:33:51Z

            ZEPPELIN-1595. Make ZeppelinContext extensible


            githubbot ASF GitHub Bot added a comment - GitHub user zjffdu opened a pull request: https://github.com/apache/zeppelin/pull/1574 ZEPPELIN-1595 . Make ZeppelinContext extensible What is this PR for? For now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well. What type of PR is it? [Feature | Refactoring] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1595 How should this be tested? Outline the steps to test the PR here. Screenshots (if appropriate) Questions: Does the licenses files need update? No Is there breaking changes for older versions? No Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1595 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1574.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1574 commit 0b6f7fefd275e1e3362e15fa85aed37b2f9c2007 Author: Jeff Zhang <zjffdu@apache.org> Date: 2016-10-31T07:33:51Z ZEPPELIN-1595 . Make ZeppelinContext extensible
            githubbot ASF GitHub Bot added a comment -

            Github user zjffdu closed the pull request at:

            https://github.com/apache/zeppelin/pull/1574

            githubbot ASF GitHub Bot added a comment - Github user zjffdu closed the pull request at: https://github.com/apache/zeppelin/pull/1574
            githubbot ASF GitHub Bot added a comment -

            GitHub user zjffdu reopened a pull request:

            https://github.com/apache/zeppelin/pull/1574

            ZEPPELIN-1595. Make ZeppelinContext extensible

                1. What is this PR for?
                  For now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well.
                1. What type of PR is it?
                  [Feature | Refactoring]
                1. Todos
            • [ ] - Task
                1. What is the Jira issue?
                1. How should this be tested?
                  Outline the steps to test the PR here.
                1. Screenshots (if appropriate)
                1. Questions:
            • Does the licenses files need update? No
            • Is there breaking changes for older versions? No
            • Does this needs documentation? No

            You can merge this pull request into a Git repository by running:

            $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1595

            Alternatively you can review and apply these changes as the patch at:

            https://github.com/apache/zeppelin/pull/1574.patch

            To close this pull request, make a commit to your master/trunk branch
            with (at least) the following in the commit message:

            This closes #1574


            commit 95ca5e51a458cbb930a794e108eac85c73525d76
            Author: Jeff Zhang <zjffdu@apache.org>
            Date: 2016-10-31T07:33:51Z

            ZEPPELIN-1595. Make ZeppelinContext extensible


            githubbot ASF GitHub Bot added a comment - GitHub user zjffdu reopened a pull request: https://github.com/apache/zeppelin/pull/1574 ZEPPELIN-1595 . Make ZeppelinContext extensible What is this PR for? For now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well. What type of PR is it? [Feature | Refactoring] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1595 How should this be tested? Outline the steps to test the PR here. Screenshots (if appropriate) Questions: Does the licenses files need update? No Is there breaking changes for older versions? No Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1595 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1574.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1574 commit 95ca5e51a458cbb930a794e108eac85c73525d76 Author: Jeff Zhang <zjffdu@apache.org> Date: 2016-10-31T07:33:51Z ZEPPELIN-1595 . Make ZeppelinContext extensible
            githubbot ASF GitHub Bot added a comment -

            Github user zjffdu commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            @Leemoonsoo @jongyoul @felixcheung Please help review.

            githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1574 @Leemoonsoo @jongyoul @felixcheung Please help review.
            githubbot ASF GitHub Bot added a comment -

            Github user felixcheung commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            In my mind the longer term goal of Zeppelin Context is to support communication between interpreter groups. Would we be going the opposite direction somewhat by having interpreter/interpreter group specific Zeppelin Context, and so they are not longer shared?

            githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1574 In my mind the longer term goal of Zeppelin Context is to support communication between interpreter groups. Would we be going the opposite direction somewhat by having interpreter/interpreter group specific Zeppelin Context, and so they are not longer shared?
            githubbot ASF GitHub Bot added a comment -

            Github user zjffdu commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            Do you mean ZeppelinContext would support communication between interpreter groups across JVM ? Currently ZeppelinContext only support spark interpreter, I don't see any case that it support communication between interpreter group. Maybe I miss something.

            githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1574 Do you mean ZeppelinContext would support communication between interpreter groups across JVM ? Currently ZeppelinContext only support spark interpreter, I don't see any case that it support communication between interpreter group. Maybe I miss something.
            githubbot ASF GitHub Bot added a comment -

            Github user felixcheung commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            To clarify, it doesn't right now, but I recall requests for that capability (which would be very useful for some of the more restricting interpreters to pass data around)

            I will let others to comment?

            githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1574 To clarify, it doesn't right now, but I recall requests for that capability (which would be very useful for some of the more restricting interpreters to pass data around) I will let others to comment?
            githubbot ASF GitHub Bot added a comment -

            Github user zjffdu closed the pull request at:

            https://github.com/apache/zeppelin/pull/1574

            githubbot ASF GitHub Bot added a comment - Github user zjffdu closed the pull request at: https://github.com/apache/zeppelin/pull/1574
            githubbot ASF GitHub Bot added a comment -

            GitHub user zjffdu reopened a pull request:

            https://github.com/apache/zeppelin/pull/1574

            ZEPPELIN-1595. Make ZeppelinContext extensible

                1. What is this PR for?
                  For now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well. For now, user need to implement the following 3 methods to extend ZeppelinContext
            • public List<Class> getSupportedClasses()
            • public abstract Map<String, String> getInterpreterClassMap();
            • protected abstract String showData(Object obj);
                1. What type of PR is it?
                  [Feature | Refactoring]
                1. Todos
            • [ ] - Task
                1. What is the Jira issue?
                1. How should this be tested?
                  Outline the steps to test the PR here.
                1. Screenshots (if appropriate)
                1. Questions:
            • Does the licenses files need update? No
            • Is there breaking changes for older versions? No
            • Does this needs documentation? No

            You can merge this pull request into a Git repository by running:

            $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1595

            Alternatively you can review and apply these changes as the patch at:

            https://github.com/apache/zeppelin/pull/1574.patch

            To close this pull request, make a commit to your master/trunk branch
            with (at least) the following in the commit message:

            This closes #1574


            commit d05a2cd4f6a1b48ae0386b77e6b5c84b6c9eb619
            Author: Jeff Zhang <zjffdu@apache.org>
            Date: 2016-10-31T07:33:51Z

            ZEPPELIN-1595. Make ZeppelinContext extensible


            githubbot ASF GitHub Bot added a comment - GitHub user zjffdu reopened a pull request: https://github.com/apache/zeppelin/pull/1574 ZEPPELIN-1595 . Make ZeppelinContext extensible What is this PR for? For now, ZeppelinContext only support Spark Interpreter. I'd like to make it extensible, so that it can support other interpreters as well. For now, user need to implement the following 3 methods to extend ZeppelinContext public List<Class> getSupportedClasses() public abstract Map<String, String> getInterpreterClassMap(); protected abstract String showData(Object obj); What type of PR is it? [Feature | Refactoring] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1595 How should this be tested? Outline the steps to test the PR here. Screenshots (if appropriate) Questions: Does the licenses files need update? No Is there breaking changes for older versions? No Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1595 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1574.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1574 commit d05a2cd4f6a1b48ae0386b77e6b5c84b6c9eb619 Author: Jeff Zhang <zjffdu@apache.org> Date: 2016-10-31T07:33:51Z ZEPPELIN-1595 . Make ZeppelinContext extensible
            githubbot ASF GitHub Bot added a comment -

            Github user wesm commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            In trying to investigate why the Apache Travis CI build queue is extremely backlogged, I found many build requests from this particular PR in https://travis-ci.org/apache/zeppelin/pull_requests

            Since this project's build matrix has 11 entries, this can pretty easily jam up the build queue for other projects. I created https://issues.apache.org/jira/browse/INFRA-13469 in an attempt to mitigate this problem in the future

            githubbot ASF GitHub Bot added a comment - Github user wesm commented on the issue: https://github.com/apache/zeppelin/pull/1574 In trying to investigate why the Apache Travis CI build queue is extremely backlogged, I found many build requests from this particular PR in https://travis-ci.org/apache/zeppelin/pull_requests Since this project's build matrix has 11 entries, this can pretty easily jam up the build queue for other projects. I created https://issues.apache.org/jira/browse/INFRA-13469 in an attempt to mitigate this problem in the future
            githubbot ASF GitHub Bot added a comment -

            Github user felixcheung commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            thanks @wesm for investigating this and following up. it's odd since this PR is 3 months old? though I'll add we should streamline the test matrix a bit (as we have from time to time)

            githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1574 thanks @wesm for investigating this and following up. it's odd since this PR is 3 months old? though I'll add we should streamline the test matrix a bit (as we have from time to time)
            githubbot ASF GitHub Bot added a comment -

            Github user wesm commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            @felixcheung it looks like these builds were triggered by a merge on a different branch. see for example https://github.com/apache/zeppelin/commit/e259feacf78770b5dc412c0642216acdce17008f

            in looking through your build history, it seems there is quite a bit of merging going on, and you may have been inadvertently triggering a lot of builds because of a quirk in Travis CI and GitHub's hooks: https://github.com/apache/zeppelin/commit/c4cf1abbf06ddf4447dacc886060ea4c1238464e

            I would recommend advising the Zeppelin contributors to avoid using `git merge` altogether because of this and prefer instead to rebase or cherry-pick.

            githubbot ASF GitHub Bot added a comment - Github user wesm commented on the issue: https://github.com/apache/zeppelin/pull/1574 @felixcheung it looks like these builds were triggered by a merge on a different branch. see for example https://github.com/apache/zeppelin/commit/e259feacf78770b5dc412c0642216acdce17008f in looking through your build history, it seems there is quite a bit of merging going on, and you may have been inadvertently triggering a lot of builds because of a quirk in Travis CI and GitHub's hooks: https://github.com/apache/zeppelin/commit/c4cf1abbf06ddf4447dacc886060ea4c1238464e I would recommend advising the Zeppelin contributors to avoid using `git merge` altogether because of this and prefer instead to rebase or cherry-pick.
            githubbot ASF GitHub Bot added a comment -

            Github user zjffdu commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            Thanks @wesm , I updated this PR yesterday, but didn't use `git merge`. I use `git rebase` and `git commit --amend`, does these also the issue as well ?

            githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1574 Thanks @wesm , I updated this PR yesterday, but didn't use `git merge`. I use `git rebase` and `git commit --amend`, does these also the issue as well ?
            githubbot ASF GitHub Bot added a comment -

            Github user wesm commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            Whatever you did generated a merge commit. The proof is right there – a rebate or amend would not have two parents

            githubbot ASF GitHub Bot added a comment - Github user wesm commented on the issue: https://github.com/apache/zeppelin/pull/1574 Whatever you did generated a merge commit. The proof is right there – a rebate or amend would not have two parents
            githubbot ASF GitHub Bot added a comment -

            Github user zjffdu commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            @Leemoonsoo @felixcheung I update this PR, please help help review that. My purpose is to make ZeppelinContext extensible is in 2 perspective.

            • All the interpreter can has its own ZeppelinContext. e.g. My next PR would to implement FlinkZeppelinContext which would display flink DataSet/Table data in zeppelin's table format
            • Interpreters can also add customized features for its own ZeppelinContext. e.g. SparkZeppelinContext can add one extention point to allow user to visualize mllib model in zeppelin.

            Communication between interpreter group mentioned by @felixcheung above is not implemented in this PR as I think this would bring much large change and need more sophisticated design. But we could put it in `BaseZeppelinContext` in future when we want to implement it.

            githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1574 @Leemoonsoo @felixcheung I update this PR, please help help review that. My purpose is to make ZeppelinContext extensible is in 2 perspective. All the interpreter can has its own ZeppelinContext. e.g. My next PR would to implement FlinkZeppelinContext which would display flink DataSet/Table data in zeppelin's table format Interpreters can also add customized features for its own ZeppelinContext. e.g. SparkZeppelinContext can add one extention point to allow user to visualize mllib model in zeppelin. Communication between interpreter group mentioned by @felixcheung above is not implemented in this PR as I think this would bring much large change and need more sophisticated design. But we could put it in `BaseZeppelinContext` in future when we want to implement it.
            githubbot ASF GitHub Bot added a comment -

            Github user zjffdu commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            @felixcheung @Leemoonsoo Would you mind to review this ?

            githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1574 @felixcheung @Leemoonsoo Would you mind to review this ?
            githubbot ASF GitHub Bot added a comment -

            Github user Leemoonsoo commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            Great work. Looks good to me!

            githubbot ASF GitHub Bot added a comment - Github user Leemoonsoo commented on the issue: https://github.com/apache/zeppelin/pull/1574 Great work. Looks good to me!
            githubbot ASF GitHub Bot added a comment -

            Github user zjffdu commented on the issue:

            https://github.com/apache/zeppelin/pull/1574

            Thanks for review, will merge if no more comments.

            githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1574 Thanks for review, will merge if no more comments.
            githubbot ASF GitHub Bot added a comment -

            Github user asfgit closed the pull request at:

            https://github.com/apache/zeppelin/pull/1574

            githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/1574

            People

              zjffdu Jeff Zhang
              zjffdu Jeff Zhang
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: