[ZEPPELIN-1325] Better Python interpreter impl

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.7.0
• Component/s: python-interpreter
• Assignee: Alexander Bezzubov
• Reporter: Alexander Bezzubov
• Created: 13/Aug/16 03:29
• Updated: 06/Jun/18 10:14

Description

We have basic Python interpreter implementation, but there are number issues that are very hard to address keeping it's existing architecture.

Without changing user-facing APIs (like `z.show`, `%python.sql` etc) the goal here is to have a more stable and generic basic Python interpreter implementation, that

• accept all valid python syntax ZEPPELIN-1133
• have accurate ZEPPELIN-1114 and fast ZEPPELIN-1360 runtime error detection
• support windows platform ZEPPELIN-893
• provides ResourcePool support (`z.get` & `z.set`)

The approach I'm exploring is: to use custom python script that

exec(paragraph)

in the Python process and communicates to Interpreter process, instead of just reading\writing to stdin\stout of the default repl.

Issue Links

• contains: ZEPPELIN-1133 python interpreter can not handle functions in certain format
• contains: ZEPPELIN-893 (Windows) Python interpreter : Support interrupt with Ms Windows
• contains: ZEPPELIN-1114 Python: improve paragraph ERROR status detection
• contains: ZEPPELIN-1360 Printing long strings is very slow in the python interpreter
• relates to: ZEPPELIN-2119 Python interpreter prints sets of dots after comments
• links to: GitHub Pull Request #1495