[ZEPPELIN-4078] Ipython improvement

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.8.1
• Component/s: python-interpreter

Environment

tested in linux / macos. Tested the ipython server directly.

• Assignee: marc hurabielle
• Reporter: marc hurabielle
• Created: 18/Mar/19 09:22
• Updated: 26/Mar/19 10:40

Description

Ipython / ipython server has currently mutliple problem. 

• Concurrent execution and auto complete can make a paragraph hang forever (until restart of the ipython server). Maybe related to https://github.com/jupyter/jupyter_client/issues/429
• Out of memory for ipython will make a paragraph hang instead of throw one error.
• High cpu usage. The loop that read from the pub/sub should not try to read everytimes the pub sub. Need to be debounce.

Overall most of those bug might be related also to some jupyter_client bug or wrong usage.
However, those are the action item:

• synchronize auto complete / paragraph execution for now
• check the kernel status when we are checking for the thread
• sleep time to time when there is no message in pub/sub
• Use only one queue instead of 3 ?

Issue Links

• links to: GitHub Pull Request #3336
• links to: GitHub Pull Request #3337
• links to: GitHub Pull Request #3339