[BEAM-13388] [beam_PostCommit_Python38] [apache_beam.ml.gcp.cloud_dlp_it_test.CloudDLPIT.test_deidentification] Failure summary

• Type: Bug
• Status: Resolved
• Priority: P1
• Resolution: Fixed
• Fix Version/s: 2.35.0
• Component/s: test-failures
• Labels: currently-failing
• Assignee: Valentyn Tymofieiev
• Reporter: Yichi Zhang
• Created: 06/Dec/21 18:27
• Updated: 23/Dec/21 05:38
• Resolved: 23/Dec/21 05:38

Description

Use this form to file an issue for test failure:

• Jenkins Job
• [Gradle Build Scan|TODO]
• [Test source code|TODO]

Initial investigation:

E         apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E         Traceback (most recent call last):
E           File "apache_beam/runners/common.py", line 1239, in apache_beam.runners.common.DoFnRunner._invoke_lifecycle_method
E           File "apache_beam/runners/common.py", line 465, in apache_beam.runners.common.DoFnInvoker.invoke_setup
E           File "/usr/local/lib/python3.8/site-packages/apache_beam/ml/gcp/cloud_dlp.py", line 193, in setup
E             'parent': self.client.project_path(self.project)
E         AttributeError: 'DlpServiceClient' object has no attribute 'project_path'
E         
E         During handling of the above exception, another exception occurred:
E         
E         Traceback (most recent call last):
E           File "/usr/local/lib/python3.8/site-packages/dataflow_worker/batchworker.py", line 644, in do_work
E             work_executor.execute()
E           File "/usr/local/lib/python3.8/site-packages/dataflow_worker/executor.py", line 208, in execute
E             op.start()
E           File "apache_beam/runners/worker/operations.py", line 699, in apache_beam.runners.worker.operations.DoOperation.start
E           File "apache_beam/runners/worker/operations.py", line 701, in apache_beam.runners.worker.operations.DoOperation.start
E           File "apache_beam/runners/worker/operations.py", line 702, in apache_beam.runners.worker.operations.DoOperation.start
E           File "apache_beam/runners/worker/operations.py", line 300, in apache_beam.runners.worker.operations.Operation.start
E           File "apache_beam/runners/worker/operations.py", line 306, in apache_beam.runners.worker.operations.Operation.start
E           File "apache_beam/runners/worker/operations.py", line 648, in apache_beam.runners.worker.operations.DoOperation.setup
E           File "apache_beam/runners/worker/operations.py", line 697, in apache_beam.runners.worker.operations.DoOperation.setup
E           File "apache_beam/runners/common.py", line 1245, in apache_beam.runners.common.DoFnRunner.setup
E           File "apache_beam/runners/common.py", line 1241, in apache_beam.runners.common.DoFnRunner._invoke_lifecycle_method
E           File "apache_beam/runners/common.py", line 1281, in apache_beam.runners.common.DoFnRunner._reraise_augmented
E           File "apache_beam/runners/common.py", line 1239, in apache_beam.runners.common.DoFnRunner._invoke_lifecycle_method
E           File "apache_beam/runners/common.py", line 465, in apache_beam.runners.common.DoFnInvoker.invoke_setup
E           File "/usr/local/lib/python3.8/site-packages/apache_beam/ml/gcp/cloud_dlp.py", line 193, in setup
E             'parent': self.client.project_path(self.project)
E         AttributeError: 'DlpServiceClient' object has no attribute 'project_path' [while running 'MaskDetectedDetails/ParDo(_DeidentifyFn)']
 

Seems google-cloud-dlp package has backward uncompatible change.

 

(Add any investigation notes so far)

---

After you've filled out the above details, please assign the issue to an individual. Assignee should treat test failures as high-priority, helping to fix the issue or find a more appropriate owner. See Apache Beam Post-Commit Policies.

Issue Links

• is duplicated by: BEAM-13433 beam_PostCommit_Python37 failing, potentially due to apache_beam.ml.gcp.cloud_dlp_it_test.CloudDLPIT
• links to: GitHub Pull Request #16142
• links to: GitHub Pull Request #16164
• links to: GitHub Pull Request #16165
• links to: GitHub Pull Request #16236
• links to: GitHub Pull Request #16268
• links to: GitHub Pull Request #16301