Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-17124

The PyFlink Job runs into infinite loop if the Python UDF imports job code

    XMLWordPrintableJSON

    Details

      Description

      If the UDF file imports job code directly or indirectly, the PyFlink Job will run into a infinite loop as follows:

      • submit job
      • execute job
      • launch UDF worker
      • import UDF
      • (If the job file is depended by UDF or imported as the top level module) import job code
      • (If the job code is executed outside the "if _name_ == '_main_':") launch gateway server and submit job to local executor
      • execute job in local mode
      • launch UDF worker
      • import UDF
      • import job code
        ...
        This infinite loop will create new Java processes and Python processes endlessly until the resources on the machine are exhausted. We should fix it ASAP.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hxbks2ks Huang Xingbo
                Reporter:
                hxbks2ks Huang Xingbo
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m