Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
2.6.1
-
None
Description
STR
- Deployed cluster with Ambari version: 2.4.3.0-30 and HDP version: 2.5.5.0-157
- Upgrade Ambari to 2.6.1.0-43
- Start EU to 2.6.4.0-36 and downgrade at final step
Result
Observed during downgrade that Livy server start failed with below error:
Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/livy_server.py", line 141, in <module> LivyServer().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute method(env) File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 970, in restart self.start(env, upgrade_type=upgrade_type) File "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/livy_server.py", line 61, in start self.configure(env) File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 120, in locking_configure original_configure(obj, *args, **kw) File "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/livy_server.py", line 51, in configure setup_livy(env, 'server', upgrade_type=upgrade_type, action = 'config') File "/var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/setup_livy.py", line 56, in setup_livy mode=0644, File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 120, in action_create raise Fail("Applying %s failed, parent directory %s doesn't exist" % (self.resource, dirname)) resource_management.core.exceptions.Fail: Applying File['/usr/hdp/current/livy-server/conf/livy-env.sh'] failed, parent directory /usr/hdp/current/livy-server/conf doesn't exist
On livy node, observed the following:
root@ctr-e134-1499953498516-324773-01-000007:/usr/hdp/current/livy-server# ls -la /usr/hdp/current/livy-server/ total 40 drwxr-xr-x 8 root root 4096 Nov 20 23:49 . drwxr-xr-x 42 root root 4096 Nov 21 00:05 .. drwxr-xr-x 2 root root 4096 Nov 20 23:43 bin lrwxrwxrwx 1 root root 14 Apr 21 2017 conf -> /etc/livy/conf drwxr-xr-x 3 root root 4096 Nov 20 23:43 etc drwxr-xr-x 2 root root 12288 Nov 20 23:43 jars drwxr-xr-x 2 livy livy 4096 Nov 20 23:49 logs drwxr-xr-x 2 root root 4096 Nov 20 23:43 repl-jars drwxr-xr-x 2 root root 4096 Nov 20 23:43 rsc-jars =============== root@ctr-e134-1499953498516-324773-01-000007:~# ls -la /etc/livy/ total 16 drwxr-xr-x 4 root root 4096 Nov 21 04:32 . drwxr-xr-x 120 root root 4096 Nov 21 03:26 .. drwxr-xr-x 3 root root 4096 Nov 21 03:30 2.6.4.0-36 lrwxrwxrwx 1 root root 21 Nov 21 04:32 conf -> /usr/hdp/current/livy drwxr-xr-x 2 root root 4096 Nov 20 23:49 conf.backup
Looks like there is no directory '/etc/livy/2.5.5.0-157/0'
From downgrade wizard, found that 'Set Version on All Hosts' Upgrade group printed the following for livy:
2017-11-21 04:32:08,022 - Link['/etc/livy/conf'] {'to': '/usr/hdp/current/livy'} 2017-11-21 04:32:08,022 - Link['/etc/livy/conf'] replacing old symlink to /grid/0/hdp/current/livy 2017-11-21 04:32:08,022 - Warning: linking to nonexistent location /usr/hdp/current/livy 2017-11-21 04:32:08,022 - Creating symbolic Link['/etc/livy/conf'] to /usr/hdp/current/livy
The cause of this appears to be that livy doesn't create a conf directory as expected:
Skipping the conf-select tool on livy since /etc/livy/conf does not exist.
Attachments
Attachments
Issue Links
- links to
