Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-21624

HDFS restart failed post Ambari upgrade when lzo compression was enabled

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.5.2
    • 2.5.2
    • ambari-server
    • None

    Description

      This is specifically for the IOP to HDP migration. We started with an IOP 4.2.5 cluster where HDFS had "com.hadoop.compression.lzo" as part of the core-site/io.compression.codecs property value. After Ambari upgrade (build 2.5.2-155), we couldn't restart HDFS, where it failed with the following error (bold/italic for the error line)

      IOP 4.2.5 lzo package is hadoop-lzo-0.5.1-2.x86_64.rpm, which doesn't fit the naming convention Ambari 2.5.2 uses (seems to be coming from HDP 2.3 stack via inheritance). It's also puzzling why the inheritance as this was right after Ambari upgrade where the stack was still IOP 4.2.5 stack that extends common services directly.

      2017-07-31 09:00:59,827 - Package['lzo'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
      2017-07-31 09:01:00,355 - Skipping installation of existing package lzo
      2017-07-31 09:01:00,355 - Package['hadoop-lzo-native'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
      2017-07-31 09:01:00,381 - Skipping installation of existing package hadoop-lzo-native
      2017-07-31 09:01:00,381 - Package['hadooplzo_4_2_5_IBM_1_1'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
      *_2017-07-31 09:01:00,407 - Installing package hadooplzo_4_2_5_IBM_1_1 ('/usr/bin/yum -d 0 -e 0 -y install hadooplzo_4_2_5_IBM_1_1')
      2017-07-31 09:01:01,212 - Execution of '/usr/bin/yum -d 0 -e 0 -y install hadooplzo_4_2_5_IBM_1_1' returned 1. Error: Nothing to do_*
      2017-07-31 09:01:01,212 - Failed to install package hadooplzo_4_2_5_IBM_1_1. Executing '/usr/bin/yum clean metadata'
      2017-07-31 09:01:01,553 - Retrying to install package hadooplzo_4_2_5_IBM_1_1 after 30 seconds
      
      Command failed after 1 tries
      
      core-site has io.compression.codecs = org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.SnappyCodec,com.hadoop.compression.lzo.LzoCodec
      

      Attachments

        1. AMBARI-21624.patch
          273 kB
          Attila Doroszlai

        Issue Links

          Activity

            People

              adoroszlai Attila Doroszlai
              dili Di Li
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: