[FLINK-31524] StreamDependencyTests.test_add_python_file failed

• Type: Technical Debt
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.17.0
• Component/s: API / Python
• Labels: stale-assigned test-stability
• Assignee: Xingbo Huang
• Reporter: Matthias Pohl
• Created: 20/Mar/23 07:28
• Updated: 12/Aug/23 10:35

Description

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=47318&view=logs&j=821b528f-1eed-5598-a3b4-7f748b13f261&t=6bb545dd-772d-5d8c-f258-f5085fba3295&l=24508

Being caused by a "No module named 'test_dependency_manage_lib'" error:

[...]
Mar 18 05:42:33 E                   Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction 1: Traceback (most recent call last):
Mar 18 05:42:33 E                     File "/__w/1/s/flink-python/.tox/py37-cython/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 287, in _execute
Mar 18 05:42:33 E                       response = task()
Mar 18 05:42:33 E                     File "/__w/1/s/flink-python/.tox/py37-cython/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 360, in <lambda>
Mar 18 05:42:33 E                       lambda: self.create_worker().do_instruction(request), request)
Mar 18 05:42:33 E                     File "/__w/1/s/flink-python/.tox/py37-cython/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 597, in do_instruction
Mar 18 05:42:33 E                       getattr(request, request_type), request.instruction_id)
Mar 18 05:42:33 E                     File "/__w/1/s/flink-python/.tox/py37-cython/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 634, in process_bundle
Mar 18 05:42:33 E                       bundle_processor.process_bundle(instruction_id))
Mar 18 05:42:33 E                     File "/__w/1/s/flink-python/.tox/py37-cython/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1004, in process_bundle
Mar 18 05:42:33 E                       element.data)
Mar 18 05:42:33 E                     File "/__w/1/s/flink-python/.tox/py37-cython/lib/python3.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 227, in process_encoded
Mar 18 05:42:33 E                       self.output(decoded_value)
Mar 18 05:42:33 E                     File "apache_beam/runners/worker/operations.py", line 526, in apache_beam.runners.worker.operations.Operation.output
Mar 18 05:42:33 E                     File "apache_beam/runners/worker/operations.py", line 528, in apache_beam.runners.worker.operations.Operation.output
Mar 18 05:42:33 E                     File "apache_beam/runners/worker/operations.py", line 237, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
Mar 18 05:42:33 E                     File "apache_beam/runners/worker/operations.py", line 240, in apache_beam.runners.worker.operations.SingletonElementConsumerSet.receive
Mar 18 05:42:33 E                     File "pyflink/fn_execution/beam/beam_operations_fast.pyx", line 169, in pyflink.fn_execution.beam.beam_operations_fast.FunctionOperation.process
Mar 18 05:42:33 E                     File "pyflink/fn_execution/beam/beam_operations_fast.pyx", line 196, in pyflink.fn_execution.beam.beam_operations_fast.FunctionOperation.process
Mar 18 05:42:33 E                     File "/__w/1/s/flink-python/pyflink/fn_execution/table/operations.py", line 110, in process_element
Mar 18 05:42:33 E                       return self.func(value)
Mar 18 05:42:33 E                     File "<string>", line 1, in <lambda>
Mar 18 05:42:33 E                     File "/__w/1/s/flink-python/pyflink/table/tests/test_dependency.py", line 51, in plus_two
Mar 18 05:42:33 E                       from test_dependency_manage_lib import add_two
Mar 18 05:42:33 E                   ModuleNotFoundError: No module named 'test_dependency_manage_lib'
Mar 18 05:42:33 E                   
Mar 18 05:42:33 E                   	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
Mar 18 05:42:33 E                   	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
Mar 18 05:42:33 E                   	at org.apache.beam.sdk.util.MoreFutures.get(MoreFutures.java:61)
Mar 18 05:42:33 E                   	at org.apache.beam.runners.fnexecution.control.SdkHarnessClient$BundleProcessor$ActiveBundle.close(SdkHarnessClient.java:504)
Mar 18 05:42:33 E                   	at org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$SimpleStageBundleFactory$1.close(DefaultJobBundleFactory.java:555)
Mar 18 05:42:33 E                   	at org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner.finishBundle(BeamPythonFunctionRunner.java:414)
Mar 18 05:42:33 E                   	... 7 more
Mar 18 05:42:33 E                   Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction 1: Traceback (most recent call last):
Mar 18 05:42:33 E                     File "/__w/1/s/flink-python/.tox/py37-cython/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 287, in _execute
Mar 18 05:42:33 E                       response = task()
Mar 18 05:42:33 E                     File "/__w/1/s/flink-python/.tox/py37-cython/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 360, in <lambda>
Mar 18 05:42:33 E                       lambda: self.create_worker().do_instruction(request), request)