[ZEPPELIN-2144] Python Interpreter although shows jobs are in running state , jobs are dependent on each other to finish

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.6.0
• Component/s: python-interpreter

Environment

Linux , Anaconda Python 2.7

• Assignee: Unassigned
• Reporter: vamsi
• Created: 21/Feb/17 17:57
• Updated: 21/Feb/17 17:57

Description

Hi,

I have setup Python interpreter connected to Anaconda Python2, initially jobs when submitted where waiting in FIFO and got to know that Zeppelin provides two scheduler modes.
1 FIFO
2.Parallel

So , I made the changes in PythonInterpreter.java to use SchedulerFactory.singleton().createOrGetParallelScheduler instead of FIFO.

The change though schedules jobs concurrently , I observe that when a long running jobs is scheduled it hold other smaller jobs which ideally should finish with no dependency ex. user 1 schedules %python
import scipy
scipy.test()

user 2 schedules %python
print ("Hello World")

The job run by user 2 waits till job of user1 complete.

Please advise.

Thanks
Vamsi