[AMBARI-21624] HDFS restart failed post Ambari upgrade when lzo compression was enabled

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.5.2
• Fix Version/s: 2.5.2
• Component/s: ambari-server
• Epic Link: Cross Stack Name Upgrades
• Assignee: Attila Doroszlai
• Reporter: Di Li
• Created: 01/Aug/17 18:09
• Updated: 02/Aug/17 14:06
• Resolved: 02/Aug/17 13:30

Description

This is specifically for the IOP to HDP migration. We started with an IOP 4.2.5 cluster where HDFS had "com.hadoop.compression.lzo" as part of the core-site/io.compression.codecs property value. After Ambari upgrade (build 2.5.2-155), we couldn't restart HDFS, where it failed with the following error (bold/italic for the error line)

IOP 4.2.5 lzo package is hadoop-lzo-0.5.1-2.x86_64.rpm, which doesn't fit the naming convention Ambari 2.5.2 uses (seems to be coming from HDP 2.3 stack via inheritance). It's also puzzling why the inheritance as this was right after Ambari upgrade where the stack was still IOP 4.2.5 stack that extends common services directly.

2017-07-31 09:00:59,827 - Package['lzo'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-07-31 09:01:00,355 - Skipping installation of existing package lzo
2017-07-31 09:01:00,355 - Package['hadoop-lzo-native'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-07-31 09:01:00,381 - Skipping installation of existing package hadoop-lzo-native
2017-07-31 09:01:00,381 - Package['hadooplzo_4_2_5_IBM_1_1'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
*_2017-07-31 09:01:00,407 - Installing package hadooplzo_4_2_5_IBM_1_1 ('/usr/bin/yum -d 0 -e 0 -y install hadooplzo_4_2_5_IBM_1_1')
2017-07-31 09:01:01,212 - Execution of '/usr/bin/yum -d 0 -e 0 -y install hadooplzo_4_2_5_IBM_1_1' returned 1. Error: Nothing to do_*
2017-07-31 09:01:01,212 - Failed to install package hadooplzo_4_2_5_IBM_1_1. Executing '/usr/bin/yum clean metadata'
2017-07-31 09:01:01,553 - Retrying to install package hadooplzo_4_2_5_IBM_1_1 after 30 seconds

Command failed after 1 tries

core-site has io.compression.codecs = org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.SnappyCodec,com.hadoop.compression.lzo.LzoCodec

Issue Links

• links to: Review request