[BEAM-7737] Microbenchmark script do not work consistently

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.14.0
• Component/s: sdk-py-harness
• Assignee: Rakesh Kumar
• Reporter: Rakesh Kumar
• Created: 13/Jul/19 18:33
• Updated: 16/May/20 13:38
• Resolved: 15/Aug/19 21:32

Description

microbenchmar scripts do not work consistently, at least run into problem for me:

(beam) ➜  python git:(master) python -m apache_beam.tools.distribution_counter_microbenchmark
Traceback (most recent call last):
  File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Users/rkumar/personal/opensource/beam/sdks/python/apache_beam/tools/distribution_counter_microbenchmark.py", line 69, in <module>
    'apache_beam.transforms.cy_dataflow_distribution_counter')
  File "apache_beam/tools/utils.py", line 41, in check_compiled
    "Profiling uncompiled code.\n"
RuntimeError: Profiling uncompiled code.
To compile beam, run 'pip install Cython; python setup.py build_ext --inplace'

Tihs is because this line of check_compiled method
doesn't work consistently. It every time just get the root module instead of the submodule and fails to identify the the given moudle is compiled with cython.
Also it is not recommended to use _import_ method because it is python internal and it is meant to be used by the Python interpreter. importlib.import_module is the recommended way of finding the module.

Pull request: https://github.com/apache/beam/pull/9066

Issue Links

• links to: GitHub Pull Request #9066