[AMBARI-26000] Ambari supports Python3 environment

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Assignee: Jialiang Cai
• Reporter: wenweijian
• Created: 03/Sep/23 15:13
• Updated: 11/Feb/24 07:34
• Resolved: 09/Feb/24 02:45

Description

This patch is about upgrade python3 on trunk branch.

All unit tests are passed on python3 version.
We also deploy it in the production environment(Centos7) and it works fine.

Upgrade Instructions:

All major changes related to this upgrade are documented in the MD file of this issue. Colleagues who need to review and understand the upgrade changes can download and review it.

1. Required Dependencies:
   After upgrading to Python 3, Ambari requires the 'distro' dependency. To install it, run:

pip3 install distro 

   If you are using CentOS 7, manually install the following dependencies:

yum install python3 python3-libs python36-distro python36-rpm 

2. Compilation and Testing:
   To compile and package, use the following Maven command:

mvn -B clean install package rpm:rpm -Dbuild-rpm -Drat.skip=true -DskipTests -Dmaven.test.skip=true 

   To run unit tests for Ambari server, use:

mvn -am test -pl ambari-server -DskipSurefireTests -Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip -Dcheckstyle.skip 

   To run unit tests for Ambari agent, use:

mvn -am test -pl ambari-agent -DskipSurefireTests -Dmaven.test.failure.ignore -Dmaven.artifact.threads=10 -Drat.skip -Dcheckstyle.skip

   Note: Before compiling, ensure that you create a symbolic link from 'python3' to 'python':

rm -f /usr/bin/python
ln -s /usr/bin/python-3.x /usr/bin/python

   This is necessary because the rpm-maven-plugin used by Ambari for RPM packaging defaults to using Python 2, and it cannot be configured to use Python 3. This symbolic link change may affect the use of 'yum.' To address this, modify the first line in the following files on CentOS 7:

/usr/libexec/urlgrabber-ext-down
/usr/bin/yum
Change '#!/usr/bin/python' to '#!/usr/bin/python2'

3. Testing:
   All unit tests have passed, and we have tested manual installation and blueprint testing in the production environment, including common component operations.

4. How to Test Installation:
   Currently, the trunk supports testing installation on CentOS 7. Pre-install the Python 3 dependencies on CentOS 7:

yum install python3 python3-libs python36-distro python36-rpm

   The installation steps are the same as before.

Installation Note:

Ambari trunk defaults to using the Apache Bigtop stack during installation, which relies on the bigtop-select tool. This tool currently defaults to Python 2, but we have updated it to Python 3 for testing purposes.

Below are the Ambari RPMs required for trunk testing. All Big Data component RPMs (packaged using Apache Bigtop) are available for CentOS 7 since trunk currently only supports CentOS 7. Download links are provided below.

http://64.69.37.12:8089/

We plan to support more operating systems after upgrading to Python 3, which will make multi-OS support easier.
 

Issue Links

• is related to: AMBARI-26004 Support Ambari agent test case execution on Python3 environment
• relates to: BIGTOP-4034 Add python3 support in bigtop-select
• links to: GitHub PR#3751