[BEAM-7099] Performance tests environment setup failed on new Jenkins nodes

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: Not applicable
• Component/s: build-system
• Assignee: Mark Liu
• Reporter: Mark Liu
• Created: 17/Apr/19 19:01
• Updated: 24/Jul/20 19:53
• Resolved: 22/Apr/19 17:55

Description

This setup failure affects all performance tests. Error in pip install step shows:

00:00:22.530 Collecting functools32 (from -r /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/PerfKitBenchmarker/requirements.txt (line 23))
00:00:22.552 Using cached https://files.pythonhosted.org/packages/c5/60/6ac26ad05857c601308d8fb9e87fa36d0ebf889423f47c3502ef034365db/functools32-3.2.3-2.tar.gz
00:00:22.772 Complete output from command python setup.py egg_info:
00:00:22.772 This backport is for Python 2.7 only.
00:00:22.772
00:00:22.772 ----------------------------------------
00:00:22.820 Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-lq5qnijw/functools32/

It installs a 2.7 only package in non-2.7 environment. Since virtualenv is installed by pip3 on new Jenkins nodes, the default py version is python3:

00:00:53.794 + virtualenv /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Python/env/.perfkit_env
00:00:57.531 Using base prefix '/usr'
00:00:57.531 New python executable in /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Python/env/.perfkit_env/bin/python3
00:00:57.531 Also creating executable in /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Python/env/.perfkit_env/bin/python
00:00:57.531 Installing setuptools, pip, wheel...
00:01:06.062 done.

Need to specify python2.7 in this case. Fix in https://github.com/apache/beam/pull/8336.