[ZEPPELIN-2197] Interpreter Lifecycle Manager

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.7.0, 0.7.1, 0.7.2, 0.8.0
• Fix Version/s: 0.8.0
• Component/s: Core, Interpreters
• Labels: features
• Assignee: Jeff Zhang
• Reporter: Ruslan Dautkhanov
• Created: 01/Mar/17 04:23
• Updated: 28/Oct/17 06:44
• Resolved: 28/Oct/17 00:09

Description

It would be great to have ability to set Interpreter idle timeout - globally or per interpreter. So if interpreter isn't used for X amount of time, it'll be killed.

Some of our users leave their Zeppelin instances idle and not used for days - that's great that they can jump right where they left off. Although in many cases we would like to kill just the interpreter processes (so the Zeppelin itself is still running).

For example, for Spark Interpreter it also leaves idle spark applications that is consuming yarn resources. We do use yarn dynamic allocation but can't set spark.dynamicAllocation.minExecutors to 0.

In our cases idle timeout for interpreters should be somewhere between 12 and 24 hours. So if a specific interpreter process was idle (not used) for that time, it'll be terminated.

As a nice side effect of this feature, if idle timeout timer will be set in an interpreter process itself, it may also help in issues like ZEPPELIN-1832 .

Thoughts?

Issue Links

• is blocked by: ZEPPELIN-2627 Umbrella for Refactoring Interpreter Component
• links to: GitHub Pull Request #2631