Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.1
    • Fix Version/s: 0.7.0, 0.8.0
    • Component/s: Interpreters
    • Labels:
      None

      Description

      With default config, Livy sessions are cleaned out after an hour of activity. This setting is controlled with livy.server.session.timeout (takes time in millisecond)

      Upon Livy session timeout, Zeppelin's LivyInterpreter needs to be restarted. This is painful user experience.

      This is a normal usage scenario & restarting LivyInterpreter should not be required.

      One possible solution is that LivyInterpreter can create a new LivySession if none exists for the same current user.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user spektom opened a pull request:

          https://github.com/apache/zeppelin/pull/1447

          ZEPPELIN-1293 Re-create Livy session if it's lost

              1. What is this PR for?
                This patch restarts Livy session when it's lost.
              1. What type of PR is it?
                Improvement/Bug Fix
              1. Todos
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1293
              1. How should this be tested?
          • Configure `livy.server.session.timeout` parameter in Livy
          • Start new Livy session through Zeppelin
          • Wait for Livy session to expire
          • See that new Livy session is started when running something in Zeppelin
              1. Questions:
          • Does the licenses files need update? - No
          • Is there breaking changes for older versions? - No
          • Does this needs documentation? - No

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/spektom/zeppelin ZEPPELIN-1293

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1447.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1447


          commit acdecbe3b1a91f630d17f4c25c6657cb3fc8d751
          Author: Michael Spector <spektom@gmail.com>
          Date: 2016-09-21T17:36:18Z

          ZEPPELIN-1293 Re-create Livy session if it's lost


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user spektom opened a pull request: https://github.com/apache/zeppelin/pull/1447 ZEPPELIN-1293 Re-create Livy session if it's lost What is this PR for? This patch restarts Livy session when it's lost. What type of PR is it? Improvement/Bug Fix Todos What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1293 How should this be tested? Configure `livy.server.session.timeout` parameter in Livy Start new Livy session through Zeppelin Wait for Livy session to expire See that new Livy session is started when running something in Zeppelin Questions: Does the licenses files need update? - No Is there breaking changes for older versions? - No Does this needs documentation? - No You can merge this pull request into a Git repository by running: $ git pull https://github.com/spektom/zeppelin ZEPPELIN-1293 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1447.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1447 commit acdecbe3b1a91f630d17f4c25c6657cb3fc8d751 Author: Michael Spector <spektom@gmail.com> Date: 2016-09-21T17:36:18Z ZEPPELIN-1293 Re-create Livy session if it's lost
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          @spektom Thanks for the contribution, I am working on a PR about integration test of livy interpreter, could you mind to wait for that PR and then add integration test to that ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1447 @spektom Thanks for the contribution, I am working on a PR about integration test of livy interpreter, could you mind to wait for that PR and then add integration test to that ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user spektom commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          Of course, it would be great.
          On Fri, Sep 23, 2016 at 06:28 Jeff Zhang <notifications@github.com> wrote:

          > @spektom <https://github.com/spektom> Thanks for the contribution, I am
          > working on a PR about integration test of livy interpreter, could you mind
          > to wait for that PR and then add integration test to that ?
          >
          > —
          > You are receiving this because you were mentioned.
          > Reply to this email directly, view it on GitHub
          > <https://github.com/apache/zeppelin/pull/1447#issuecomment-249093645>, or mute
          > the thread
          > <https://github.com/notifications/unsubscribe-auth/AAJpO0VBwVJ9NayJbPq3h9j1bRji_4cuks5qs0dkgaJpZM4KDF0H>
          > .
          >

          Show
          githubbot ASF GitHub Bot added a comment - Github user spektom commented on the issue: https://github.com/apache/zeppelin/pull/1447 Of course, it would be great. On Fri, Sep 23, 2016 at 06:28 Jeff Zhang <notifications@github.com> wrote: > @spektom < https://github.com/spektom > Thanks for the contribution, I am > working on a PR about integration test of livy interpreter, could you mind > to wait for that PR and then add integration test to that ? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < https://github.com/apache/zeppelin/pull/1447#issuecomment-249093645 >, or mute > the thread > < https://github.com/notifications/unsubscribe-auth/AAJpO0VBwVJ9NayJbPq3h9j1bRji_4cuks5qs0dkgaJpZM4KDF0H > > . >
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          @spektom and @zjffdu is it possible to add a null check also. I've been doing some debugging over the past few days and certain situations can cause nulls to be returned and in theory if a null is being returned the session is dead.

          *So instead of this:*
          if (json.matches("^(\")?Session (\'[0-9]\' )?not found(.?\"?)$")) {
          throw new LivyNoSessionException();

          *This:*
          boolean clearSession = false;
          if (json != null) {
          if (json.matches("^(\")?Session (\'[0-9]\' )?not found(.?\"?)$"))

          { clearSession = true; }

          } else

          { clearSession = true; }

          if (clearSession)

          { throw new LivyNoSessionException(); }
          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1447 @spektom and @zjffdu is it possible to add a null check also. I've been doing some debugging over the past few days and certain situations can cause nulls to be returned and in theory if a null is being returned the session is dead. * So instead of this: * if (json.matches("^(\")?Session (\' [0-9] \' )?not found(.?\"?)$")) { throw new LivyNoSessionException(); * This: * boolean clearSession = false; if (json != null) { if (json.matches("^(\")?Session (\' [0-9] \' )?not found(.?\"?)$")) { clearSession = true; } } else { clearSession = true; } if (clearSession) { throw new LivyNoSessionException(); }
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          Also I have some concerns on using Exceptions to as goto's as there is definite performance issues that occur by using exceptions... http://javarevisited.blogspot.com/2013/03/0-exception-handling-best-practices-in-Java-Programming.html. Specifically since code inside of the try catch cannot be optimized.

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1447 Also I have some concerns on using Exceptions to as goto's as there is definite performance issues that occur by using exceptions... http://javarevisited.blogspot.com/2013/03/0-exception-handling-best-practices-in-Java-Programming.html . Specifically since code inside of the try catch cannot be optimized.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user spektom commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          @gss2002 you shouldn't be concerned about performance as this code only runs when command is executed.

          Show
          githubbot ASF GitHub Bot added a comment - Github user spektom commented on the issue: https://github.com/apache/zeppelin/pull/1447 @gss2002 you shouldn't be concerned about performance as this code only runs when command is executed.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          @spektom this fix is good. Did some extensive load testing with it this AM and it solves the session expiration issues. Thanks for the contribution

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1447 @spektom this fix is good. Did some extensive load testing with it this AM and it solves the session expiration issues. Thanks for the contribution
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          @spektom / @zjffdu @purechoc there is definitely an additional condition. Not sure if it's because the ConcurrentHashMaps are not being used correctly. But the exception doesn't get caught completely or correctly at times with the fix proposed here..

          ERROR [2016-10-19 14:19:57,638] (

          {pool-2-thread-11} LivyHelper.java[executeHTTP]:378) - Error with 404 StatusCode: "Session '9' not found."
          ERROR [2016-10-19 14:19:57,638] ({pool-2-thread-11}

          LivyHelper.java[interpretInput]:229) - error in interpretInput
          org.apache.zeppelin.livy.LivyHelper$LivyNoSessionException: Session not found, Livy server would have restarted, or lost session.
          at org.apache.zeppelin.livy.LivyHelper.executeCommand(LivyHelper.java:312)
          at org.apache.zeppelin.livy.LivyHelper.interpret(LivyHelper.java:241)
          at org.apache.zeppelin.livy.LivyHelper.interpretInput(LivyHelper.java:189)
          at org.apache.zeppelin.livy.LivySparkInterpreter.interpret(LivySparkInterpreter.java:106)
          at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
          at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:390)
          at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
          at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)
          INFO [2016-10-19 14:19:57,639] (

          {pool-2-thread-11}

          SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1476901197622 finished by scheduler org.apache.zeppelin.livy.LivySparkInterpreter37814848
          INFO [2016-10-19 14:19:57,819] (

          {pool-2-thread-34} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1476901197819 started by scheduler org.apache.zeppelin.livy.LivySparkInterpreter37814848
          ERROR [2016-10-19 14:19:57,835] ({pool-2-thread-34}

          LivyHelper.java[executeHTTP]:378) - Error with 404 StatusCode: "Session '9' not found."
          ERROR [2016-10-19 14:19:57,835] (

          {pool-2-thread-34} LivyHelper.java[interpretInput]:229) - error in interpretInput
          org.apache.zeppelin.livy.LivyHelper$LivyNoSessionException: Session not found, Livy server would have restarted, or lost session.
          at org.apache.zeppelin.livy.LivyHelper.executeCommand(LivyHelper.java:312)
          at org.apache.zeppelin.livy.LivyHelper.interpret(LivyHelper.java:241)
          at org.apache.zeppelin.livy.LivyHelper.interpretInput(LivyHelper.java:189)
          at org.apache.zeppelin.livy.LivySparkInterpreter.interpret(LivySparkInterpreter.java:106)
          at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
          at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:390)
          at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
          at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)
          INFO [2016-10-19 14:19:57,836] ({pool-2-thread-34}

          SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1476901197819 finished by scheduler org.apache.zeppelin.livy.LivySparkInterpreter37814848

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1447 @spektom / @zjffdu @purechoc there is definitely an additional condition. Not sure if it's because the ConcurrentHashMaps are not being used correctly. But the exception doesn't get caught completely or correctly at times with the fix proposed here.. ERROR [2016-10-19 14:19:57,638] ( {pool-2-thread-11} LivyHelper.java [executeHTTP] :378) - Error with 404 StatusCode: "Session '9' not found." ERROR [2016-10-19 14:19:57,638] ({pool-2-thread-11} LivyHelper.java [interpretInput] :229) - error in interpretInput org.apache.zeppelin.livy.LivyHelper$LivyNoSessionException: Session not found, Livy server would have restarted, or lost session. at org.apache.zeppelin.livy.LivyHelper.executeCommand(LivyHelper.java:312) at org.apache.zeppelin.livy.LivyHelper.interpret(LivyHelper.java:241) at org.apache.zeppelin.livy.LivyHelper.interpretInput(LivyHelper.java:189) at org.apache.zeppelin.livy.LivySparkInterpreter.interpret(LivySparkInterpreter.java:106) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:390) at org.apache.zeppelin.scheduler.Job.run(Job.java:176) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) INFO [2016-10-19 14:19:57,639] ( {pool-2-thread-11} SchedulerFactory.java [jobFinished] :137) - Job remoteInterpretJob_1476901197622 finished by scheduler org.apache.zeppelin.livy.LivySparkInterpreter37814848 INFO [2016-10-19 14:19:57,819] ( {pool-2-thread-34} SchedulerFactory.java [jobStarted] :131) - Job remoteInterpretJob_1476901197819 started by scheduler org.apache.zeppelin.livy.LivySparkInterpreter37814848 ERROR [2016-10-19 14:19:57,835] ({pool-2-thread-34} LivyHelper.java [executeHTTP] :378) - Error with 404 StatusCode: "Session '9' not found." ERROR [2016-10-19 14:19:57,835] ( {pool-2-thread-34} LivyHelper.java [interpretInput] :229) - error in interpretInput org.apache.zeppelin.livy.LivyHelper$LivyNoSessionException: Session not found, Livy server would have restarted, or lost session. at org.apache.zeppelin.livy.LivyHelper.executeCommand(LivyHelper.java:312) at org.apache.zeppelin.livy.LivyHelper.interpret(LivyHelper.java:241) at org.apache.zeppelin.livy.LivyHelper.interpretInput(LivyHelper.java:189) at org.apache.zeppelin.livy.LivySparkInterpreter.interpret(LivySparkInterpreter.java:106) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:390) at org.apache.zeppelin.scheduler.Job.run(Job.java:176) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) INFO [2016-10-19 14:19:57,836] ({pool-2-thread-34} SchedulerFactory.java [jobFinished] :137) - Job remoteInterpretJob_1476901197819 finished by scheduler org.apache.zeppelin.livy.LivySparkInterpreter37814848
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user spektom commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          @gss2002 are you using latest code from the branch? I can't find org.apache.zeppelin.livy.LivyHelper.interpretInput(LivyHelper.java:189) invocation at org.apache.zeppelin.livy.LivySparkInterpreter.interpret(LivySparkInterpreter.java:106) as it's shown in your stack trace.

          Show
          githubbot ASF GitHub Bot added a comment - Github user spektom commented on the issue: https://github.com/apache/zeppelin/pull/1447 @gss2002 are you using latest code from the branch? I can't find org.apache.zeppelin.livy.LivyHelper.interpretInput(LivyHelper.java:189) invocation at org.apache.zeppelin.livy.LivySparkInterpreter.interpret(LivySparkInterpreter.java:106) as it's shown in your stack trace.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          @spektom I think what happens here is this code fires.. which has nothing to do with the fix here..

          in LivySparkInterpreter:
          return livyHelper.interpretInput(line, interpreterContext, userSessionMap, out,
          sessionId2AppIdMap.get(sessionId), sessionId2WebUIMap.get(sessionId), displayAppInfo);

          That gets called before the NoSessionException occurs.. And then in
          LivyHelper --> public InterpreterResult interpretInput grabs the exception and handles it. I guess the question is can we do a rootcause on this and rethrow?

          } catch (Exception e)

          { LOGGER.error("error in interpretInput", e); return new InterpreterResult(Code.ERROR, e.getMessage()); }
          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1447 @spektom I think what happens here is this code fires.. which has nothing to do with the fix here.. in LivySparkInterpreter: return livyHelper.interpretInput(line, interpreterContext, userSessionMap, out, sessionId2AppIdMap.get(sessionId), sessionId2WebUIMap.get(sessionId), displayAppInfo); That gets called before the NoSessionException occurs.. And then in LivyHelper --> public InterpreterResult interpretInput grabs the exception and handles it. I guess the question is can we do a rootcause on this and rethrow? } catch (Exception e) { LOGGER.error("error in interpretInput", e); return new InterpreterResult(Code.ERROR, e.getMessage()); }
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          @spektom just tested against my build by catching the exception and rethrowing.. It definitely solves the issue.

          } catch (LivyNoSessionException e)

          { throw e; }

          catch (Exception e)

          { LOGGER.error("error in interpretInput", e); return new InterpreterResult(Code.ERROR, e.getMessage()); }

          }

          public InterpreterResult interpret(String stringLines,

          The code base I'm using is here with you patch and a few of @zjffdu patches and one of mine for NestedRuntimeException for 404's with KerberosTemplate: https://github.com/gss2002/zeppelin/blob/GSS_PROD_BUILD/livy/src/main/java/org/apache/zeppelin/livy/LivyHelper.java

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1447 @spektom just tested against my build by catching the exception and rethrowing.. It definitely solves the issue. } catch (LivyNoSessionException e) { throw e; } catch (Exception e) { LOGGER.error("error in interpretInput", e); return new InterpreterResult(Code.ERROR, e.getMessage()); } } public InterpreterResult interpret(String stringLines, The code base I'm using is here with you patch and a few of @zjffdu patches and one of mine for NestedRuntimeException for 404's with KerberosTemplate: https://github.com/gss2002/zeppelin/blob/GSS_PROD_BUILD/livy/src/main/java/org/apache/zeppelin/livy/LivyHelper.java
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          Sorry for late response. @spektom Could you add some additional message of session recreation to be displayed in frontend. Because if session is recreated, we may need to rerun all the paragraphs, without knowing session recreationusers may be confused for errors like some variables are not defined.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1447 Sorry for late response. @spektom Could you add some additional message of session recreation to be displayed in frontend. Because if session is recreated, we may need to rerun all the paragraphs, without knowing session recreationusers may be confused for errors like some variables are not defined.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user purechoc commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          LGTM. it's working nicely in my environment. (zeppelin 0.7 + livy 2.1)

          Show
          githubbot ASF GitHub Bot added a comment - Github user purechoc commented on the issue: https://github.com/apache/zeppelin/pull/1447 LGTM. it's working nicely in my environment. (zeppelin 0.7 + livy 2.1)
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          @purechoc Do you want to rebase it and continue work on it ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1447 @purechoc Do you want to rebase it and continue work on it ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user zjffdu opened a pull request:

          https://github.com/apache/zeppelin/pull/1861

          ZEPPELIN-1293. Livy Interpreter: Automatically attach or create to a new session

              1. What is this PR for?
                By default, livy session will expire after one hour. This PR would create session automatically for user if session is expired, and would also display the session expire information in frontend. The expire message would only display at the first time of session recreation, after that the message won't be displayed.
              1. What type of PR is it?
                [Improvement ]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
              1. How should this be tested?
                Tested manually.
                ![2017-01-06_1903](https://cloud.githubusercontent.com/assets/164491/21716129/7fdab630-d443-11e6-9901-39130c3b1caf.png)
              1. Screenshots (if appropriate)
              1. Questions:
          • Does the licenses files need update? No
          • Is there breaking changes for older versions? No
          • Does this needs documentation? No

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1293

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1861.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1861


          commit 852f9829df58dce0dfb1ee78460884722a170b2b
          Author: Jeff Zhang <zjffdu@apache.org>
          Date: 2017-01-06T11:05:38Z

          ZEPPELIN-1293. Livy Interpreter: Automatically attach or create to a new session


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user zjffdu opened a pull request: https://github.com/apache/zeppelin/pull/1861 ZEPPELIN-1293 . Livy Interpreter: Automatically attach or create to a new session What is this PR for? By default, livy session will expire after one hour. This PR would create session automatically for user if session is expired, and would also display the session expire information in frontend. The expire message would only display at the first time of session recreation, after that the message won't be displayed. What type of PR is it? [Improvement ] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1293 How should this be tested? Tested manually. ! [2017-01-06_1903] ( https://cloud.githubusercontent.com/assets/164491/21716129/7fdab630-d443-11e6-9901-39130c3b1caf.png ) Screenshots (if appropriate) Questions: Does the licenses files need update? No Is there breaking changes for older versions? No Does this needs documentation? No You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1293 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1861.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1861 commit 852f9829df58dce0dfb1ee78460884722a170b2b Author: Jeff Zhang <zjffdu@apache.org> Date: 2017-01-06T11:05:38Z ZEPPELIN-1293 . Livy Interpreter: Automatically attach or create to a new session
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1447

          @spektom Sorry, I didn't get your reply, I create another PR for it #1861

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1447 @spektom Sorry, I didn't get your reply, I create another PR for it #1861
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1861

          @prabhjyotsingh @felixcheung Please help review.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1861 @prabhjyotsingh @felixcheung Please help review.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1861

          @felixcheung I have addressed the comments and also fix one bug in LivySparkSQLInterpreter, and update the screenshot of testing in PR description.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1861 @felixcheung I have addressed the comments and also fix one bug in LivySparkSQLInterpreter, and update the screenshot of testing in PR description.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user prabhjyotsingh commented on the issue:

          https://github.com/apache/zeppelin/pull/1861

          Tested locally, looks good.

          Show
          githubbot ASF GitHub Bot added a comment - Github user prabhjyotsingh commented on the issue: https://github.com/apache/zeppelin/pull/1861 Tested locally, looks good.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on the issue:

          https://github.com/apache/zeppelin/pull/1861

          LGTM.
          what is the bug in LivySparkSQLInterpreter - should we have a JIRA for it?

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1861 LGTM. what is the bug in LivySparkSQLInterpreter - should we have a JIRA for it?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1861

          The bug in LivySparkSQLInterpreter is caused by this PR. LivySparkSQLInterpreter delegate LivySparkInterpreter to generate InterpreterResult and then convert it to table type. So here since we make it as multiple InterpreterResult, we should only convert the InterpreterMessage with text type that is for sql output.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1861 The bug in LivySparkSQLInterpreter is caused by this PR. LivySparkSQLInterpreter delegate LivySparkInterpreter to generate InterpreterResult and then convert it to table type. So here since we make it as multiple InterpreterResult, we should only convert the InterpreterMessage with text type that is for sql output.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1861

          ping @felixcheung any more comments ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1861 ping @felixcheung any more comments ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user felixcheung commented on the issue:

          https://github.com/apache/zeppelin/pull/1861

          nope,
          merging if no more comment

          Show
          githubbot ASF GitHub Bot added a comment - Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1861 nope, merging if no more comment
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/zeppelin/pull/1861

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/1861
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1861

          @felixcheung @minahlee Could any of you help to merge it into branch-0.7 as well ? Since this is only livy interpreter specific, so I don't think it would affect the 0.7 release.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1861 @felixcheung @minahlee Could any of you help to merge it into branch-0.7 as well ? Since this is only livy interpreter specific, so I don't think it would affect the 0.7 release.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/1861

          @felixcheung @minahlee , Sorry my mistake, I has been merged into branch-0.7. @minahlee I notice you make the jira fixversion as 0.8, I add 0.7 as well.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1861 @felixcheung @minahlee , Sorry my mistake, I has been merged into branch-0.7. @minahlee I notice you make the jira fixversion as 0.8, I add 0.7 as well.

            People

            • Assignee:
              zjffdu Jeff Zhang
              Reporter:
              vinayshukla@gmail.com Vinay Shukla
            • Votes:
              5 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development