[SAMZA-890] ThreadFactory used by HttpServer should use Daemon threads

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0
• Fix Version/s: 0.10.1
• Assignee: Jagadish
• Reporter: Jagadish
• Created: 10/Mar/16 08:00
• Updated: 14/Mar/16 06:17
• Resolved: 14/Mar/16 06:17

Description

The Samza HttpServer.scala encapsulates a Jetty Http Server. The default constructor of the Jetty Http Server includes a QueuedThreadPool of Non-Daemon threads. This has the following problem:

1. If there is a bug/ or an error in the SamzaAppMaster and the SamzaAppMaster is shutting down, the threads in the QueuedThreadPool stay alive. The process does not terminate as a result. So, even subtle bugs that cause AppMaster failures will be magnified because the process will not terminate and merely hang. Yarn's RM will not restart the process because the process is still alive.

The fix is to ensure that the QueuedThreadPool creates Daemon threads by setting isDaemon to true.

Here's an example of Apache Calcite's HttpServer handling this:

http://www.programcreek.com/java-api-examples/index.php?source_dir=calcite-master/spark/src/main/java/org/apache/calcite/adapter/spark/HttpServer.java