Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6384

PythonStreamer does not close python processes

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.3.0
    • Component/s: Python API
    • Labels:
      None

      Description

      The PythonStreamer opens a new process calling the python binary to check whether the binary is available. This process won't get closed leading to an excessive number of open python processes when running the PythonPlanBinderTest. I'm not entirely sure whether we need this extra process, because the actual python call with the python code would also fail if there is no python binary available.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user tillrohrmann opened a pull request:

          https://github.com/apache/flink/pull/3774

          FLINK-6384 [py] Remove python binary check via additional process

          The PythonStreamer used to check for the existence of the python binary by
          starting a python process. This process was not closed afterwards. This caused
          the PythonPlanBinderTest to fail locally.

          I think the check whether a python binary exists is not necessary since the
          subsequent python command would fail anyway if there is no binary available on
          the system. The system failure message is that there is no such file or directory.
          This error message should be descriptive enough in order to debug such a problem.

          cc @zentol

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/tillrohrmann/flink fixPythonStreamer

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3774.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3774


          commit 746fe59737338c02502443c5c3e88c613d0b92ad
          Author: Till Rohrmann <trohrmann@apache.org>
          Date: 2017-04-25T18:41:58Z

          FLINK-6384 [py] Remove python binary check via additional process

          The PythonStreamer used to check for the existence of the python binary by
          starting a python process. This process was not closed afterwards. This caused
          the PythonPlanBinderTest to fail locally.

          I think the check whether a python binary exists is not necessary since the
          subsequent python command would fail anyway if there is no binary available on
          the system. The system failure message is that there is no such file or directory.
          This error message should be descriptive enough in order to debug such a problem.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user tillrohrmann opened a pull request: https://github.com/apache/flink/pull/3774 FLINK-6384 [py] Remove python binary check via additional process The PythonStreamer used to check for the existence of the python binary by starting a python process. This process was not closed afterwards. This caused the PythonPlanBinderTest to fail locally. I think the check whether a python binary exists is not necessary since the subsequent python command would fail anyway if there is no binary available on the system. The system failure message is that there is no such file or directory. This error message should be descriptive enough in order to debug such a problem. cc @zentol You can merge this pull request into a Git repository by running: $ git pull https://github.com/tillrohrmann/flink fixPythonStreamer Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3774.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3774 commit 746fe59737338c02502443c5c3e88c613d0b92ad Author: Till Rohrmann <trohrmann@apache.org> Date: 2017-04-25T18:41:58Z FLINK-6384 [py] Remove python binary check via additional process The PythonStreamer used to check for the existence of the python binary by starting a python process. This process was not closed afterwards. This caused the PythonPlanBinderTest to fail locally. I think the check whether a python binary exists is not necessary since the subsequent python command would fail anyway if there is no binary available on the system. The system failure message is that there is no such file or directory. This error message should be descriptive enough in order to debug such a problem.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/3774

          Looks good, +1 to merge.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/3774 Looks good, +1 to merge.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user tillrohrmann commented on the issue:

          https://github.com/apache/flink/pull/3774

          Thanks for the review @zentol. The failing test cases are unrelated. Merging this PR.

          Show
          githubbot ASF GitHub Bot added a comment - Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/3774 Thanks for the review @zentol. The failing test cases are unrelated. Merging this PR.
          Hide
          till.rohrmann Till Rohrmann added a comment -

          Fixed via 1829819b64fb32740bcbd4d5074e3fa138276f89

          Show
          till.rohrmann Till Rohrmann added a comment - Fixed via 1829819b64fb32740bcbd4d5074e3fa138276f89
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3774

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3774

            People

            • Assignee:
              till.rohrmann Till Rohrmann
              Reporter:
              till.rohrmann Till Rohrmann
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development