[IMPALA-8508] Use Python 3 from toolchain for impala-python

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Infrastructure
• Target Version: Impala 4.3.0
• Epic Color: ghx-label-9
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 06/May/19 17:53
• Updated: 28/Feb/23 17:35

Description

We should standardise on a single python version to use for tests and other infrastructure. Python 2.7 is going EOL soon.

I started adding it to the toolchain - https://gerrit.cloudera.org/#/c/14161/

Issue Links

• is depended upon by: IMPALA-11955 Support building without Python 2
• relates to: IMPALA-9627 Update Impala utility Python scripts to be Python3 compatible

Sub-Tasks

• Test impala-shell distribution instead of special dev environment version: Resolved
• Use Python 2.7 from toolchain: Resolved
• Add a script that compiles the python code with Python 2 and Python 3: Resolved
• Fix all syntax errors for Python 3: Resolved
• Update toolchain python 3 to 3.7.16: Resolved
• Add impala-python3 virtualenv: Resolved
• Add absolute_import and division __future__ imports to all python code: Resolved
• Fix range/xrange/map/filter/ifilter/zip/izip issues regarding iterators being lazy: Resolved
• Fix dictionary methods that became lazy: Resolved
• Fix use of deprecated functions / packages in Python3 (long, maxint, file, etc): Resolved
• Fix broken imports on Python 3 and Python 3 object model differences: Resolved
• Implement Unicode sandwich for python code: Open
• Impala-shell should switch to absolute imports: Open
• Switch report_benchmark_results.py to Python 3: Resolved
• Install thrift into the impala-python virtualenv rather than using the toolchain: Resolved
• Add a way to run tests with python 3: Resolved
• Migrate tests that use the 'unittest' package to use normal pytest base class: Resolved
• Set of tests for exploration_strategy=exhaustive varies between python 2 and 3: Open
• Drop dependency on platform.linux_distribution to support Python 3.8: Resolved