[SAMZA-2198] Container process always takes task.shutdown.ms to shut down - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 1.0, 1.1
Fix Version/s: 1.3
Component/s: None
Labels:
None

Description

https://issues.apache.org/jira/browse/SAMZA-1724 added a System.exit to LocalContainerRunner (now moved to ContainerLaunchUtil) in order to ensure the process exits. However, this seems to cause a partial deadlock when shutting down the container process. SamzaContainer adds a shutdown hook which joins on the run loop thread (which is also the main thread). During the shutdown process, when the main thread reaches the System.exit in ContainerLaunchUtil, it looks like it waits for the shutdown hook to complete. However, the shutdown hook is waiting for the main thread to join. Therefore, the shutdown hook will always wait task.shutdown.ms before giving up on joining the main thread, and then the process exits.

This issue was not observed in a job on 0.14. The System.exit was only added in 1.0.