Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-2198

Container process always takes task.shutdown.ms to shut down

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0, 1.1
    • Fix Version/s: 1.3
    • Component/s: None
    • Labels:
      None

      Description

      https://issues.apache.org/jira/browse/SAMZA-1724 added a System.exit to LocalContainerRunner (now moved to ContainerLaunchUtil) in order to ensure the process exits. However, this seems to cause a partial deadlock when shutting down the container process. SamzaContainer adds a shutdown hook which joins on the run loop thread (which is also the main thread). During the shutdown process, when the main thread reaches the System.exit in ContainerLaunchUtil, it looks like it waits for the shutdown hook to complete. However, the shutdown hook is waiting for the main thread to join. Therefore, the shutdown hook will always wait task.shutdown.ms before giving up on joining the main thread, and then the process exits.

      This issue was not observed in a job on 0.14. The System.exit was only added in 1.0.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                abkshvn Abhishek Shivanna
                Reporter:
                cameronlee Cameron Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h
                  2h