Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7873

FnApi with Subprocess runner hangs frequently when running with multi workers with py2

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • P0
    • Resolution: Fixed
    • None
    • 2.15.0
    • sdk-py-core
    • None

    Description

      Pipeline hangs at subprocess.Popen() when shut it down. I looked into source code of subprocess lib. py27 doesn't do any lock while py3 locks when waiting. Py3 added locks at other places of Popen() as well, all unlocked places with py2 may contribute to the problem. We can add a lock when calling Popen() to prevent the deadlock. 

      Attachments

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-3645 Support multi-process execution on the FnApiRunner

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Triage Needed
          links to

          Web Link GitHub Pull Request #9218

          Activity

            People

              hannahjiang Hannah Jiang
              hannahjiang Hannah Jiang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: