[ZEPPELIN-2724] Zeppelin gets stuck with java.lang.Thread.State: BLOCKED (on object monitor)

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.7.2, 0.8.0
• Component/s: zeppelin-server
• Assignee: Unassigned
• Reporter: Sergey Mazin
• Created: 05/Jul/17 14:01
• Updated: 29/Nov/17 11:11

Description

We are using Zeppelin in 10+ active users environment. At least 2-3 times per day Zeppelin hangs with no errors in log files.

I checked Jstack and looks like Zeppelin is waiting for lock on some resource
"qtp905654280-2488" #2488 prio=5 os_prio=0 tid=0x00007fcbacfa9000 nid=0x3a00 waiting for monitor entry [0x00007fcb826a7000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.zeppelin.socket.NotebookServer.removeConnectionFromAllNote(NotebookServer.java:449)

• waiting to lock <0x000000068006d060> (a java.util.HashMap)
  at org.apache.zeppelin.socket.NotebookServer.sendHomeNote(NotebookServer.java:851)
  at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:241)
  at org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:59)

which is locked by other thread.

at org.apache.zeppelin.socket.NotebookSocket.send(NotebookSocket.java:72)

• locked <0x000000079e7d7b88> (a org.apache.zeppelin.socket.NotebookSocket)
  at org.apache.zeppelin.socket.NotebookServer.broadcast(NotebookServer.java:495)
• locked <0x000000068006d060> (a java.util.HashMap)

Issue Links

• Blocked: ZEPPELIN-2717 Thread BLOCKED after restarting an interpreter
• links to: GitHub Pull Request #2465
• links to: GitHub Pull Request #2544