[AMBARI-22547] History and Hive server start failed during IOP migration with AttributeError

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.6.1
• Fix Version/s: 2.6.1
• Assignee: Jonathan Hurley
• Reporter: Vivek Sharma
• Created: 29/Nov/17 16:50
• Updated: 29/Nov/17 22:38
• Resolved: 29/Nov/17 21:11

Description

STR

1. Deployed cluster with Ambari version: 2.4.2.0_IBM_000000 and HDP version: 4.2.5.0-0000 (LZO is enabled in HDFS configs)
2. Upgrade Ambari to Target Version: 2.6.1.0-64 | Hash: cd4db8e9ac0ea7ce14fc1253959a121688f34952
3. Register HDP-2.6.4.0-51 version and call remove iop-slect
4. Install the bits and start EU

Result
History server start failed with below error:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 129, in <module>
    HistoryServer().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 946, in restart
    self.pre_upgrade_restart(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py", line 81, in pre_upgrade_restart
    copy_to_hdfs("tez", params.user_group, params.hdfs_user, skip=params.sysprep_skip_copy_tarballs_hdfs)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 464, in copy_to_hdfs
    source_file = prepare_function()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py", line 94, in _prepare_tez_tarball
    hadoop_lib_native_lzo_dir = os.path.join(stack_root, tez_version, "hadoop", "lib", "native")
  File "/usr/lib64/python2.6/posixpath.py", line 65, in join
    if b.startswith('/'):
AttributeError: 'NoneType' object has no attribute 'startswith'

The error looks to be due to lzo changes, as the following code has the issue:

# if enabled, LZO GPL libraries must be copied as well
     91   if lzo_utils.should_install_lzo():
     92     stack_root = Script.get_stack_root()
     93     tez_version = component_version.get_component_repository_version("TEZ")
     94     hadoop_lib_native_lzo_dir = os.path.join(stack_root, tez_version, "hadoop", "lib", "native")
     95

Issue Links

• links to: ReviewBoard