[BEAM-1883] Ignore the result of shutil.rmtree

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: sdk-py-core
• Labels: newbie starter
• Assignee: Unassigned
• Reporter: Ahmet Altay
• Created: 05/Apr/17 00:42
• Updated: 03/Jun/22 17:12

Description

There is a post commit failure with failure to delete a temp directory after use. Should we ignore these errors?

https://builds.apache.org/view/Beam/job/beam_PostCommit_Python_Verify/1738/console

Traceback (most recent call last):
File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/transforms/sideinputs_test.py", line 248, in test_as_singleton_with_different_defaults
pipeline.run()
File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/test_pipeline.py", line 91, in run
result = super(TestPipeline, self).run()
File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py", line 169, in run
return self.runner.run(self)
File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", line 32, in run
self.result = super(TestDataflowRunner, self).run(pipeline)
File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", line 175, in run
self.dataflow_client.create_job(self.job), self)
File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/utils/retry.py", line 174, in wrapper
return fun(*args, **kwargs)
File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 424, in create_job
self.create_job_description(job)
File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 445, in create_job_description
job.options, file_copy=self._gcs_file_copy)
File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/dependency.py", line 411, in stage_job_resources
shutil.rmtree(temp_dir)
File "/usr/lib/python2.7/shutil.py", line 239, in rmtree
onerror(os.listdir, path, sys.exc_info())
File "/usr/lib/python2.7/shutil.py", line 237, in rmtree
names = os.listdir(path)
OSError: [Errno 2] No such file or directory: '/tmp/tmpQXRC2r'