[BEAM-8397] DataflowRunnerTest.test_remote_runner_display_data sometimes fails due to infinite recursion during pickling on Python 3.7.

• Type: Sub-task
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.17.0
• Component/s: sdk-py-core
• Assignee: Valentyn Tymofieiev
• Reporter: Valentyn Tymofieiev
• Created: 14/Oct/19 19:23
• Updated: 16/May/20 13:39
• Resolved: 31/Oct/19 17:56

Description

`python ./setup.py test -s apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest.test_remote_runner_display_data` passes.
`tox -e py37-gcp` passes if Beam depends on dill==0.3.0, but fails if Beam depends on dill==0.3.1.1.`python ./setup.py nosetests --tests 'apache_beam/runners/dataflow/dataflow_runner_test.py:DataflowRunnerTest.test_remote_runner_display_data` fails currently if run on master.

The failure indicates infinite recursion during pickling:

test_remote_runner_display_data (apache_beam.runners.dataflow.dataflow_runner_test.DataflowRunnerTest) ... Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00007f9d700ed740 (most recent call first):
  File "/usr/lib/python3.7/pickle.py", line 479 in get
  File "/usr/lib/python3.7/pickle.py", line 497 in save
  File "/usr/lib/python3.7/pickle.py", line 786 in save_tuple
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
  File "/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py", line 1394 in save_function
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 882 in _batch_setitems
  File "/usr/lib/python3.7/pickle.py", line 856 in save_dict
  File "/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py", line 910 in save_module_dict
  File "/usr/local/google/home/valentyn/projects/beam/clean/beam/sdks/python/apache_beam/internal/pickler.py", line 198 in new_save_module_dict
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 786 in save_tuple
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
  File "/usr/local/google/home/valentyn/projects/beam/clean/beam/sdks/python/apache_beam/internal/pickler.py", line 114 in wrapper
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 771 in save_tuple
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
  File "/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py", line 1137 in save_cell
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 771 in save_tuple
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 786 in save_tuple
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 638 in save_reduce
  File "/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py", line 1394 in save_function
  File "/usr/lib/python3.7/pickle.py", line 504 in save
  File "/usr/lib/python3.7/pickle.py", line 882 in _batch_setitems
  File "/usr/lib/python3.7/pickle.py", line 856 in save_dict
  File "/usr/local/google/home/valentyn/tmp/py37env/lib/python3.7/site-packages/dill/_dill.py", line 910 in save_module_dict
  File "/usr/local/google/home/valentyn/projects/beam/clean/beam/sdks/python/apache_beam/internal/pickler.py", line 198 in new_save_module_dict
...

cc: yoshiki.obata

Issue Links

• blocks: BEAM-5878 Support DoFns with Keyword-only arguments in Python 3.
• links to: GitHub Pull Request #9881