[BEAM-8152] Provide a way to better control minor+patch versions of Python 3.x interpreters used to run Beam tests locally and on Jenkins.

• Type: Sub-task
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: sdk-py-core, testing
• Assignee: Unassigned
• Reporter: Valentyn Tymofieiev
• Created: 04/Sep/19 19:12
• Updated: 09/Mar/22 00:56

Description

Currently, Beam Python test infrastructure does not provide fine-grained way to control Python interpreter version. The major+minor version are typically selected by virtual environment, and the patch version of interpreter is defined by the version of python package available on machine that is running the tests.

For example, Jenkins ubuntu-based machines, use Python 3.5.2 for python 3.5 test suites, while debian-based SDK harness containers for Python 3.5 come with Python 3.5.6, and the python3.5 package available on my dev machine is Python 3.5.4.

Throughout development of Python 3.5.x, Cpython implementation details that have changed in and these changes affect certain codepaths in Beam, such as type inference.

When we encounter such issues, it is difficult for Beam developers to test their changes against a particular patch version of Python interpreter both remotely and locally. Opening this issue to make it simpler.

cc: markflyhigh yifanzou udim altay who may have opinions and ideas about how to make this simpler.

Note that there are separate questions:
1) which patch versions of Python we should test against on Jenkins
2) which patch versions of Python Beam should claim to support.

Regardless of the answers to those questions, we may want to make it easier for an engineer to run a test suite against a particular patch version of Python, and/or make it easier to switch which patch version is used by Jenkins.

Issue Links

• links to: GitHub Pull Request #15819