[SPARK-8763] executing run-tests.py with Python 2.6 fails with absence of subprocess.check_output function

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.5.0
• Component/s: PySpark
• Labels: pyspark testing

Environment

Mac OS X 10.10.3 Python 2.6.9 Java 1.8.0

• Target Version/s: 1.5.0
• Flags: Patch
• Assignee: Tomohiko K.
• Reporter: Tomohiko K.
• Created: 01/Jul/15 15:21
• Updated: 01/Jul/15 22:07
• Resolved: 01/Jul/15 16:37

Description

Running run-tests.py with Python 2.6 cause following error:

Running PySpark tests. Output is in python//Users/tomohiko/.jenkins/jobs/pyspark_test/workspace/python/unit-tests.log
Will test against the following Python executables: ['python2.6', 'python3.4', 'pypy']
Will test the following Python modules: ['pyspark-core', 'pyspark-ml', 'pyspark-mllib', 'pyspark-sql', 'pyspark-streaming']
Traceback (most recent call last):
  File "./python/run-tests.py", line 196, in <module>
    main()
  File "./python/run-tests.py", line 159, in main
    python_implementation = subprocess.check_output(
AttributeError: 'module' object has no attribute 'check_output'
...

The cause of this error is using subprocess.check_output function, which exists since Python 2.7.
(ref. https://docs.python.org/2.7/library/subprocess.html#subprocess.check_output)

Issue Links

• links to: [Github] Pull Request #7161 (cocoatomo)