[BEAM-8966] failure in :sdks:python:test-suites:direct:py37:hdfsIntegrationTest

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: Not applicable
• Component/s: test-failures
• Assignee: Unassigned
• Reporter: Udi Meiri
• Created: 13/Dec/19 19:30
• Updated: 24/Jul/20 19:59
• Resolved: 10/Jun/20 18:39

Description

I believe this is due to https://github.com/apache/beam/pull/9915

Collecting mypy-protobuf==1.12
  Using cached https://files.pythonhosted.org/packages/b6/28/041dea47c93564bfc0ece050362894292ec4f173caa92fa82994a6d061d1/mypy_protobuf-1.12-py3-none-any.whl
Installing collected packages: mypy-protobuf
Successfully installed mypy-protobuf-1.12
beam_fn_api.proto: warning: Import google/protobuf/descriptor.proto but not used.
beam_fn_api.proto: warning: Import google/protobuf/wrappers.proto but not used.
Traceback (most recent call last):
  File "/usr/local/bin/protoc-gen-mypy", line 13, in <module>
    import google.protobuf.descriptor_pb2 as d
ModuleNotFoundError: No module named 'google'
--mypy_out: protoc-gen-mypy: Plugin failed with status code 1.
Process Process-1:
Traceback (most recent call last):
  File "/app/sdks/python/gen_protos.py", line 104, in generate_proto_files
    from grpc_tools import protoc
ModuleNotFoundError: No module named 'grpc_tools'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/app/sdks/python/gen_protos.py", line 189, in _install_grpcio_tools_and_generate_proto_files
    generate_proto_files()
  File "/app/sdks/python/gen_protos.py", line 144, in generate_proto_files
    '%s' % ret_code)
RuntimeError: Protoc returned non-zero status (see logs for details): 1
Traceback (most recent call last):
  File "/app/sdks/python/gen_protos.py", line 104, in generate_proto_files
    from grpc_tools import protoc
ModuleNotFoundError: No module named 'grpc_tools'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 295, in <module>
    'mypy': generate_protos_first(mypy),
  File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.7/site-packages/setuptools/command/sdist.py", line 44, in run
    self.run_command('egg_info')
  File "/usr/local/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 220, in run
    gen_protos.generate_proto_files(log=log)
  File "/app/sdks/python/gen_protos.py", line 121, in generate_proto_files
    raise ValueError("Proto generation failed (see log for details).")
ValueError: Proto generation failed (see log for details).
Service 'test' failed to build: The command '/bin/sh -c cd sdks/python &&     python setup.py sdist &&     pip install --no-cache-dir $(ls dist/apache-beam-*.tar.gz | tail -n1)[gcp]' returned a non-zero code: 1

https://builds.apache.org/job/beam_PostCommit_Python37/1114/consoleText