Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0, 0.8.0
    • Component/s: None
    • Labels:
      None

      Description

      Although the cancel api of LivyInterpreter is implemented, but it just cancel to poll the result, didn't cancel the real execution in livy side. So it would better to cancel the execution on livy side as well.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user zjffdu opened a pull request:

          https://github.com/apache/zeppelin/pull/1859

          ZEPPELIN-1432. Support cancellation of paragraph execution

              1. What is this PR for?
                Livy 0.3 support cancel operation, this PR is to support cancel in livy interpreter. First we would check the livy version, then based on the livy version, we would call the livy rest api to cancel the statement.
              1. What type of PR is it?
                Improvement | Feature ]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
              1. How should this be tested?
                Tested manually, because cancel is only avaible in livy 0.3 which is not released yet.
              1. Screenshots (if appropriate)
                ![image](https://cloud.githubusercontent.com/assets/164491/21712520/3ed292ec-d430-11e6-8829-581e1bba1a9c.png)
              1. Questions:
          • Does the licenses files need update? No
          • Is there breaking changes for older versions? No
          • Does this needs documentation? No

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1432

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1859.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1859


          commit f1ace29d514d15afcdfe4d0e7f0cf9984bb67829
          Author: Jeff Zhang <zjffdu@apache.org>
          Date: 2016-12-26T02:06:01Z

          ZEPPELIN-1432. Support cancellation of paragraph execution


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user zjffdu opened a pull request: https://github.com/apache/zeppelin/pull/1859 ZEPPELIN-1432 . Support cancellation of paragraph execution What is this PR for? Livy 0.3 support cancel operation, this PR is to support cancel in livy interpreter. First we would check the livy version, then based on the livy version, we would call the livy rest api to cancel the statement. What type of PR is it? Improvement | Feature ] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1432 How should this be tested? Tested manually, because cancel is only avaible in livy 0.3 which is not released yet. Screenshots (if appropriate) ! [image] ( https://cloud.githubusercontent.com/assets/164491/21712520/3ed292ec-d430-11e6-8829-581e1bba1a9c.png ) Questions: Does the licenses files need update? No Is there breaking changes for older versions? No Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1432 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1859.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1859 commit f1ace29d514d15afcdfe4d0e7f0cf9984bb67829 Author: Jeff Zhang <zjffdu@apache.org> Date: 2016-12-26T02:06:01Z ZEPPELIN-1432 . Support cancellation of paragraph execution
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          CI failure is not relevant.
          @prabhjyotsingh @felixcheung Please help review.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1859 CI failure is not relevant. @prabhjyotsingh @felixcheung Please help review.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          looks good except for a few comments.
          is it possible to add tests for these changes?

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1859 looks good except for a few comments. is it possible to add tests for these changes?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          cancel is only available in livy 0.3 which is not released yet, so no test for it. I can add that after livy 0.3 is released.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1859 cancel is only available in livy 0.3 which is not released yet, so no test for it. I can add that after livy 0.3 is released.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu closed the pull request at:

          https://github.com/apache/zeppelin/pull/1859

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu closed the pull request at: https://github.com/apache/zeppelin/pull/1859
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user zjffdu reopened a pull request:

          https://github.com/apache/zeppelin/pull/1859

          ZEPPELIN-1432. Support cancellation of paragraph execution

              1. What is this PR for?
                Livy 0.3 support cancel operation, this PR is to support cancel in livy interpreter. First we would check the livy version, then based on the livy version, we would call the livy rest api to cancel the statement.
              1. What type of PR is it?
                Improvement | Feature ]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
              1. How should this be tested?
                Tested manually, because cancel is only avaible in livy 0.3 which is not released yet.
              1. Screenshots (if appropriate)
                ![image](https://cloud.githubusercontent.com/assets/164491/21712520/3ed292ec-d430-11e6-8829-581e1bba1a9c.png)
              1. Questions:
          • Does the licenses files need update? No
          • Is there breaking changes for older versions? No
          • Does this needs documentation? No

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1432

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1859.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1859


          commit 070fea05ca9d7c3da3ac5708cf46dcec20e17a45
          Author: Jeff Zhang <zjffdu@apache.org>
          Date: 2016-12-26T02:06:01Z

          ZEPPELIN-1432. Support cancellation of paragraph execution

          commit 1cbeb2612898656bbe6b4a1d9753306839fb3ae5
          Author: Jeff Zhang <zjffdu@apache.org>
          Date: 2017-01-15T02:18:27Z

          add zeppelin.livy.pull_status.interval.millis


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user zjffdu reopened a pull request: https://github.com/apache/zeppelin/pull/1859 ZEPPELIN-1432 . Support cancellation of paragraph execution What is this PR for? Livy 0.3 support cancel operation, this PR is to support cancel in livy interpreter. First we would check the livy version, then based on the livy version, we would call the livy rest api to cancel the statement. What type of PR is it? Improvement | Feature ] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1432 How should this be tested? Tested manually, because cancel is only avaible in livy 0.3 which is not released yet. Screenshots (if appropriate) ! [image] ( https://cloud.githubusercontent.com/assets/164491/21712520/3ed292ec-d430-11e6-8829-581e1bba1a9c.png ) Questions: Does the licenses files need update? No Is there breaking changes for older versions? No Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1432 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1859.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1859 commit 070fea05ca9d7c3da3ac5708cf46dcec20e17a45 Author: Jeff Zhang <zjffdu@apache.org> Date: 2016-12-26T02:06:01Z ZEPPELIN-1432 . Support cancellation of paragraph execution commit 1cbeb2612898656bbe6b4a1d9753306839fb3ae5 Author: Jeff Zhang <zjffdu@apache.org> Date: 2017-01-15T02:18:27Z add zeppelin.livy.pull_status.interval.millis
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu closed the pull request at:

          https://github.com/apache/zeppelin/pull/1859

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu closed the pull request at: https://github.com/apache/zeppelin/pull/1859
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user zjffdu reopened a pull request:

          https://github.com/apache/zeppelin/pull/1859

          ZEPPELIN-1432. Support cancellation of paragraph execution

              1. What is this PR for?
                Livy 0.3 support cancel operation, this PR is to support cancel in livy interpreter. First we would check the livy version, then based on the livy version, we would call the livy rest api to cancel the statement.
              1. What type of PR is it?
                Improvement | Feature ]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
              1. How should this be tested?
                Tested manually, because cancel is only avaible in livy 0.3 which is not released yet.
              1. Screenshots (if appropriate)
                ![image](https://cloud.githubusercontent.com/assets/164491/21712520/3ed292ec-d430-11e6-8829-581e1bba1a9c.png)
              1. Questions:
          • Does the licenses files need update? No
          • Is there breaking changes for older versions? No
          • Does this needs documentation? No

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1432

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1859.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1859


          commit 070fea05ca9d7c3da3ac5708cf46dcec20e17a45
          Author: Jeff Zhang <zjffdu@apache.org>
          Date: 2016-12-26T02:06:01Z

          ZEPPELIN-1432. Support cancellation of paragraph execution

          commit 1cbeb2612898656bbe6b4a1d9753306839fb3ae5
          Author: Jeff Zhang <zjffdu@apache.org>
          Date: 2017-01-15T02:18:27Z

          add zeppelin.livy.pull_status.interval.millis


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user zjffdu reopened a pull request: https://github.com/apache/zeppelin/pull/1859 ZEPPELIN-1432 . Support cancellation of paragraph execution What is this PR for? Livy 0.3 support cancel operation, this PR is to support cancel in livy interpreter. First we would check the livy version, then based on the livy version, we would call the livy rest api to cancel the statement. What type of PR is it? Improvement | Feature ] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1432 How should this be tested? Tested manually, because cancel is only avaible in livy 0.3 which is not released yet. Screenshots (if appropriate) ! [image] ( https://cloud.githubusercontent.com/assets/164491/21712520/3ed292ec-d430-11e6-8829-581e1bba1a9c.png ) Questions: Does the licenses files need update? No Is there breaking changes for older versions? No Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1432 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1859.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1859 commit 070fea05ca9d7c3da3ac5708cf46dcec20e17a45 Author: Jeff Zhang <zjffdu@apache.org> Date: 2016-12-26T02:06:01Z ZEPPELIN-1432 . Support cancellation of paragraph execution commit 1cbeb2612898656bbe6b4a1d9753306839fb3ae5 Author: Jeff Zhang <zjffdu@apache.org> Date: 2017-01-15T02:18:27Z add zeppelin.livy.pull_status.interval.millis
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          ping @felixcheung @prabhjyotsingh CI failure is not relevant.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1859 ping @felixcheung @prabhjyotsingh CI failure is not relevant.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          minor question. LGTM.

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1859 minor question. LGTM.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          could you trigger CI again?

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1859 could you trigger CI again?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu closed the pull request at:

          https://github.com/apache/zeppelin/pull/1859

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu closed the pull request at: https://github.com/apache/zeppelin/pull/1859
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user zjffdu reopened a pull request:

          https://github.com/apache/zeppelin/pull/1859

          ZEPPELIN-1432. Support cancellation of paragraph execution

              1. What is this PR for?
                Livy 0.3 support cancel operation, this PR is to support cancel in livy interpreter. First we would check the livy version, then based on the livy version, we would call the livy rest api to cancel the statement.
              1. What type of PR is it?
                Improvement | Feature ]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
              1. How should this be tested?
                Tested manually, because cancel is only avaible in livy 0.3 which is not released yet.
              1. Screenshots (if appropriate)
                ![image](https://cloud.githubusercontent.com/assets/164491/21712520/3ed292ec-d430-11e6-8829-581e1bba1a9c.png)
              1. Questions:
          • Does the licenses files need update? No
          • Is there breaking changes for older versions? No
          • Does this needs documentation? No

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1432

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1859.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1859


          commit 070fea05ca9d7c3da3ac5708cf46dcec20e17a45
          Author: Jeff Zhang <zjffdu@apache.org>
          Date: 2016-12-26T02:06:01Z

          ZEPPELIN-1432. Support cancellation of paragraph execution

          commit 1cbeb2612898656bbe6b4a1d9753306839fb3ae5
          Author: Jeff Zhang <zjffdu@apache.org>
          Date: 2017-01-15T02:18:27Z

          add zeppelin.livy.pull_status.interval.millis

          commit 200ca71296669bdcd9b249f659522fec5316ff64
          Author: Jeff Zhang <zjffdu@apache.org>
          Date: 2017-01-16T00:17:53Z

          address comments

          commit 83eaf83f5bd2ee8f87664bd1f639de1a0e6ddf08
          Author: Jeff Zhang <zjffdu@apache.org>
          Date: 2017-01-16T01:01:54Z

          minor update


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user zjffdu reopened a pull request: https://github.com/apache/zeppelin/pull/1859 ZEPPELIN-1432 . Support cancellation of paragraph execution What is this PR for? Livy 0.3 support cancel operation, this PR is to support cancel in livy interpreter. First we would check the livy version, then based on the livy version, we would call the livy rest api to cancel the statement. What type of PR is it? Improvement | Feature ] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1432 How should this be tested? Tested manually, because cancel is only avaible in livy 0.3 which is not released yet. Screenshots (if appropriate) ! [image] ( https://cloud.githubusercontent.com/assets/164491/21712520/3ed292ec-d430-11e6-8829-581e1bba1a9c.png ) Questions: Does the licenses files need update? No Is there breaking changes for older versions? No Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1432 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1859.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1859 commit 070fea05ca9d7c3da3ac5708cf46dcec20e17a45 Author: Jeff Zhang <zjffdu@apache.org> Date: 2016-12-26T02:06:01Z ZEPPELIN-1432 . Support cancellation of paragraph execution commit 1cbeb2612898656bbe6b4a1d9753306839fb3ae5 Author: Jeff Zhang <zjffdu@apache.org> Date: 2017-01-15T02:18:27Z add zeppelin.livy.pull_status.interval.millis commit 200ca71296669bdcd9b249f659522fec5316ff64 Author: Jeff Zhang <zjffdu@apache.org> Date: 2017-01-16T00:17:53Z address comments commit 83eaf83f5bd2ee8f87664bd1f639de1a0e6ddf08 Author: Jeff Zhang <zjffdu@apache.org> Date: 2017-01-16T01:01:54Z minor update
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          I trigger the CI, but the last CI failure is not relevant. Because this PR only modify livy interpreter, other failure should be flaky test. Unfortunately we seems to have more and more flaky test

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1859 I trigger the CI, but the last CI failure is not relevant. Because this PR only modify livy interpreter, other failure should be flaky test. Unfortunately we seems to have more and more flaky test
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          ping @felixcheung

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1859 ping @felixcheung
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user prabhjyotsingh commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          Sure CI failure looks it is not relevant to this. LGTM!

          Show
          githubbot ASF GitHub Bot added a comment - Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1859 Sure CI failure looks it is not relevant to this. LGTM!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          that's odd
          ```
          WelcomePageSuite:
          - Welcome sign is correct *** FAILED ***
           The code passed to eventually never returned normally. Attempted 16 times over 20.46449854 seconds. Last failure message: WelcomePageSuite.this.find("welcome")(WelcomePageSuite.this.driver).isDefined was false. (WelcomePageSuite.scala:31)
          Zeppelin stop [ OK ]
          Run completed in 37 seconds, 499 milliseconds.
          Total number of tests run: 1
          Suites: completed 3, aborted 0
          Tests: succeeded 0, failed 1, canceled 0, ignored 0, pending 0
          *** 1 TEST FAILED ***
          [ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project zeppelin-server: There are test failures -> [Help 1]
          [ERROR]
          [31mException encountered when invoking run on a nested suite - The code passed to eventually never returned normally. Attempted 16 times over 20.170745813 seconds. Last failure message: AbstractFunctionalSuite.this.find("welcome")(AbstractFunctionalSuite.this.webDriver).isDefined was false. *** ABORTED ***
           Exception encountered when invoking run on a nested suite - The code passed to eventually never returned normally. Attempted 16 times over 20.170745813 seconds. Last failure message: AbstractFunctionalSuite.this.find("welcome")(AbstractFunctionalSuite.this.webDriver).isDefined was false. (AbstractFunctionalSuite.scala:44)
          Run completed in 33 seconds, 483 milliseconds.
          Total number of tests run: 0
          Suites: completed 1, aborted 1
          ```

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1859 that's odd ``` WelcomePageSuite: - Welcome sign is correct *** FAILED ***  The code passed to eventually never returned normally. Attempted 16 times over 20.46449854 seconds. Last failure message: WelcomePageSuite.this.find("welcome")(WelcomePageSuite.this.driver).isDefined was false. (WelcomePageSuite.scala:31) Zeppelin stop [ OK  [0;39m] Run completed in 37 seconds, 499 milliseconds. Total number of tests run: 1 Suites: completed 3, aborted 0 Tests: succeeded 0, failed 1, canceled 0, ignored 0, pending 0 *** 1 TEST FAILED *** [ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project zeppelin-server: There are test failures -> [Help 1] [ERROR] [31mException encountered when invoking run on a nested suite - The code passed to eventually never returned normally. Attempted 16 times over 20.170745813 seconds. Last failure message: AbstractFunctionalSuite.this.find("welcome")(AbstractFunctionalSuite.this.webDriver).isDefined was false. *** ABORTED ***  Exception encountered when invoking run on a nested suite - The code passed to eventually never returned normally. Attempted 16 times over 20.170745813 seconds. Last failure message: AbstractFunctionalSuite.this.find("welcome")(AbstractFunctionalSuite.this.webDriver).isDefined was false. (AbstractFunctionalSuite.scala:44) Run completed in 33 seconds, 483 milliseconds. Total number of tests run: 0 Suites: completed 1, aborted 1 ```
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/zeppelin/pull/1859

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/1859
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1859

          That's flaky test, I several times of `WelcomePageSuite` in CI failure.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1859 That's flaky test, I several times of `WelcomePageSuite` in CI failure.

            People

            • Assignee:
              zjffdu Jeff Zhang
              Reporter:
              zjffdu Jeff Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development