[AMBARI-14362] Express Upgrade: Kafka broker restart failed during EU from 2.2.9 to 2.3.2 with customized service user accounts

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.3.0, 2.2.0
• Component/s: ambari-server
• Assignee: Dmitry Lysnichenko
• Reporter: Dmitry Lysnichenko
• Created: 13/Dec/15 18:40
• Updated: 15/Dec/15 05:49
• Resolved: 14/Dec/15 23:47

Description

Steps:
Setup HDP 2.2.9 cluster with Ambari build #2.2.0.0-1281; the cluster has HA enabled and setup with customized service users for all services
Did EU from HDP 2.2.9 to 2.3.2.0

Result:
Observed during EU that Kakfa broker restart failed on all hosts with error:

Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py", line 101, in <module>
KafkaBroker().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 218, in execute
method(env)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 496, in restart
self.stop(env, upgrade_type=upgrade_type)
File "/var/lib/ambari-agent/cache/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py", line 88, in stop
user=params.kafka_user,
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 158, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 121, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'source /usr/hdp/current/kafka-broker/config/kafka-env.sh; /usr/hdp/current/kafka-broker/bin/kafka stop' returned 1. Stopping Kafka [23359] /usr/hdp/current/kafka-broker/bin/kafka: line 110: /var/log/kafka/kafka.out: Permission denied
failed.

The problem is likely due to custom service user being used for Kafka

Issue Links

• links to: review_request_