[ZEPPELIN-1832] Zombie Interpreter processes

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.7.0
• Fix Version/s: 0.7.1, 0.8.0
• Component/s: Core, Interpreters, zeppelin-server
• Assignee: Jongyoul Lee
• Reporter: Ruslan Dautkhanov
• Created: 16/Dec/16 16:36
• Updated: 22/Mar/17 16:39
• Resolved: 16/Mar/17 14:54

Description

When we restart Zeppelin server (main process), in many cases, Interpreter
process keeps running, essential becoming a zombie processes.

In case of Spark interpreter, it also holds SparkContext - consuming server-side resources too.

As discussed in users@ and dev@ mailing lists, other users have confirmed this problem too.
Please see [Interpreter zombie processes|http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Interpreter-zombie-processes-td4738.html.

luciano resende

I have also seen similar issues even using zeppelin-ddeamon but didn't have much time to investigate the issue when it was happening to me.

zjffdu

I believe I see this before too.

[e-mail]

Have similar experience, although hard to say what's the reason as all process supposed to killed, as Moon pointed out.
Also noticed that with `mvn tests`, after almost every run, there are 1-2 zombie RemoteInterpreter processes hanging around.

@ blaubaer <[e-mail]>

We are seeing this problem as well, regularly actually. Especially in
situations when we have many concurrent interpreters running.

How to reproduce
1. Start Zeppelin
2. Run a paragraph \w any interpreters -> RemoteInterpreterServer process will be shown up (check by ps -ef | grep RemoteInterpreterServer)
3. Stop Zeppelin
4. check ps -ef | grep RemoteInterpreterServer again

Issue Links

• blocks: ZEPPELIN-2134 Release 0.7.1
• duplicates: ZEPPELIN-1888 RemoteInterpreterServer processes are still alive even if Zeppelin server stopped
• is duplicated by: ZEPPELIN-2258 Spark driver not closed on interpreter restart
• links to: GitHub Pull Request #1838