[SPARK-21573] Tests failing with run-tests.py SyntaxError occasionally in Jenkins

• Type: Test
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.3.0
• Component/s: Tests
• Assignee: Shane Knapp
• Reporter: Hyukjin Kwon
• Created: 29/Jul/17 04:05
• Updated: 12/Dec/22 18:10
• Resolved: 01/Aug/17 22:38

Description

It looks default python in the path at few places such as ./dev/run-tests use Python 2.6 in Jenkins and it fails to execute run-tests.py:

python2.6 run-tests.py
  File "run-tests.py", line 124
    {m: set(m.dependencies).intersection(modules_to_test) for m in modules_to_test}, sort=True)
                                                            ^
SyntaxError: invalid syntax

It looks there are quite some places to fix to support Python 2.6 in run-tests.py and related Python scripts.
We might just try to set Python 2.7 in few other scripts running this if available.

Please also see http://apache-spark-developers-list.1001551.n3.nabble.com/Tests-failing-with-run-tests-py-SyntaxError-td22030.html