[AMBARI-15988] spark history server fails to start after upgrading ambari 2.2.1.0 -> 2.2.2.0 and remaining on derg

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Fix Version/s: 2.2.2
• Assignee: Andrew Onischuk
• Reporter: Andrew Onischuk
• Created: 20/Apr/16 11:44
• Updated: 25/Apr/16 18:54
• Resolved: 25/Apr/16 18:54

Description

After upgrading the cluster to Ambari 2.2.2.0 from ambari 2.2.1.0 various
services had a restart alert and upon restarting spark we saw a failure. IE
only ambari upgrade was done.

Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py", line 95, in <module>
JobHistoryServer().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py", line 55, in start
spark_service('jobhistoryserver', upgrade_type=upgrade_type, action='start')
File "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.0.2.2/package/scripts/spark_service.py", line 49, in spark_service
recursive_chmod=True
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in _init_
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 463, in action_create_on_execute
self.action_delayed("create")
File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 460, in action_delayed
self.get_hdfs_resource_executor().action_delayed(action_name, self)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 250, in action_delayed
self._assert_valid()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 234, in _assert_valid
self.target_status = self._get_file_status(target)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 295, in _get_file_status
list_status = self.util.run_command(target, 'GETFILESTATUS', method='GET', ignore_status_codes=['404'], assertable_result=False)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 182, in run_command
_, out, err = get_user_call_output(cmd, user=self.run_user, logoutput=self.logoutput, quiet=False)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py", line 61, in get_user_call_output
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'curl -sS -L -w '%

{http_code}

' -X GET --negotiate -u : 'http://hcube2-1n03.eng.hortonworks.com:50070/webhdfs/v1{{spark_history_dir}}?op=GETFILESTATUS&user.name=hdfs' 1>/tmp/tmpwmdLi2 2>/tmp/tmpH9CmmS' returned 3. curl: (3) [globbing] nested braces not supported at pos 57

Stack that is running right now is HDP 2.4.0.0

Ambari is running here:
<https://hcube2-1n01.eng.hortonworks.com:8443/#/main/dashboard/metrics>

Issue Links

• links to: ReviewBoard