[BEAM-7847] Generate Python SDK docs using Python 3

• Type: Sub-task
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: Not applicable
• Component/s: sdk-py-core
• Assignee: yoshiki obata
• Reporter: Valentyn Tymofieiev
• Created: 30/Jul/19 08:07
• Updated: 24/Jul/20 19:51
• Resolved: 08/Apr/20 17:55

Description

Currently scripts/generate_pydoc.sh script fails on Python 3 with "RuntimeError: empty_like method already has a docstring" errors:

pip install -e .[gcp,test]
pip install Sphinx==1.6.5
pip install sphinx_rtd_theme==0.2.4
./scripts/generate_pydoc.sh

/home/valentyn/projects/beam/beam/beam/sdks/python/target/docs/source/apache_beam.testing.benchmarks.nexmark.queries.query0.rst:4: WARNING: autodoc: failed to import module 'apache_beam.testing.benchmarks.nexmark.queries.query0'; the following exception was raised:
Traceback (most recent call last):
  File "/home/valentyn/tmp/venv/py3/lib/python3.6/site-packages/sphinx/ext/autodoc.py", line 658, in import_object
    __import__(self.modname)
  File "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/__init__.py", line 98, in <module>
    from apache_beam import io
  File "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/io/__init__.py", line 22, in <module>
    from apache_beam.io.avroio import *
  File "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/io/avroio.py", line 61, in <module>
    from apache_beam.io import filebasedsink
  File "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/io/filebasedsink.py", line 34, in <module>
    from apache_beam.io import iobase
  File "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/io/iobase.py", line 50, in <module>
    from apache_beam.transforms import core
  File "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/transforms/__init__.py", line 29, in <module>
    from apache_beam.transforms.util import *
  File "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/transforms/util.py", line 228, in <module>
    class _BatchSizeEstimator(object):
  File "/home/valentyn/projects/beam/beam/beam/sdks/python/apache_beam/transforms/util.py", line 359, in _BatchSizeEstimator
    import numpy as np
  File "/home/valentyn/tmp/venv/py3/lib/python3.6/site-packages/numpy/__init__.py", line 142, in <module>
    from . import core
  File "/home/valentyn/tmp/venv/py3/lib/python3.6/site-packages/numpy/core/__init__.py", line 17, in <module>
    from . import multiarray
  File "/home/valentyn/tmp/venv/py3/lib/python3.6/site-packages/numpy/core/multiarray.py", line 78, in <module>
    def empty_like(prototype, dtype=None, order=None, subok=None, shape=None):
  File "/home/valentyn/tmp/venv/py3/lib/python3.6/site-packages/numpy/core/overrides.py", line 203, in decorator
    docs_from_dispatcher=docs_from_dispatcher)(implementation)
  File "/home/valentyn/tmp/venv/py3/lib/python3.6/site-packages/numpy/core/overrides.py", line 159, in decorator
    add_docstring(implementation, dispatcher.__doc__)
RuntimeError: empty_like method already has a docstring

Issue Links

• links to: GitHub Pull Request #9804
• links to: GitHub Pull Request #10141