[AMBARI-21355] Upgrade to Ambari Server 2.5.1 breaks common-services provided by MPacks

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.5.0
• Fix Version/s: 2.5.3
• Component/s: ambari-server, ambari-upgrade

Environment

Ubuntu Linux 14.04

• Assignee: Vitaly Brodetskyi
• Reporter: David Kaspar
• Created: 27/Jun/17 12:14
• Updated: 13/Jul/17 17:44
• Resolved: 13/Jul/17 17:43

Description

How to reproduce:

• Have your own MPack that is similar to one that is located at "contrib/management-packs/microsoft-r_mpack" in Ambari source code
• Have an Ambari Server 2.4.3 with the MPack installed in a cluster
• Upgrade Ambari-Server to 2.5.1 version

Expected Behavior:
Ambari Server is upgraded and MPack is still installed and used

Actual Behavior:
When running ambari-server upgrade, it fails with the following error message:

Using python  /usr/bin/python
Upgrading ambari-server
INFO: Upgrade Ambari Server
INFO: Updating Ambari Server properties in ambari.properties ...
INFO: Updating Ambari Server properties in ambari-env.sh ...
WARNING: Original file ambari-env.sh kept
ERROR: Unexpected OSError: [Errno 17] File exists
For more info run ambari-server with -v or --verbose option

The following steps are used for Ambari-Server upgrade:

service ambari-server stop
ambari-agent stop
wget -nv http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.5.1.0/ambari.list -O /etc/apt/sources.list.d/ambari.list
apt-get clean all
apt-get update
apt-get install ambari-server
apt-get install ambari-agent
ambari-server upgrade

Analyses:
Usually when such a common service is installed from MPack, a new directory (named by the service) is created in /var/lib/ambari-server/resources/common-services/ directory. The service directory contain a single symbolic link (named by the version of the service) and targets /var/lib/ambari-server/resources/mpacks/<MY_MPACK_NAME>-<MY_MPACK_VERSION>/common-services/<MY_SERVICE_NAME>/<MY_SERVICE_VERSION>

It seems that the whole upgrade process works correctly until

ambari-server upgrade

command runs find_and_copy_custom_services function that re-creates common-services from the backup directory that is created during apt-get install ambari-server. The backup directory is correctly having the symbolic link in it while the newly created /var/lib/ambari-server/resources/common-services/ is NOT having the symbolic link. Instead it contains full-deep copy of the directory targeted by the original symbolic link.

Proposed Solution:
Replace the following line in ambari-server/src/main/python/ambari_server/serverUpgrade.py:

shutil.copytree(backup_service, os.path.join(current_base_service_dir,service_name))

with:

shutil.copytree(backup_service, os.path.join(current_base_service_dir,service_name), True)

so the service directory gets copied while keeping symbolic links.

Issue Links

• duplicates: AMBARI-21263 Ambari 2.5.1 upgrade fails with HDPSearch mpack installed