[AMBARI-19115] Metrics Monitor start failed: ERROR: Cannot write pid /grid/0/pid/metric_monitor/ambari-metrics-monitor.pid

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.5.0
• Fix Version/s: 2.5.0
• Assignee: Andrew Onischuk
• Reporter: Vivek Rathod
• Created: 06/Dec/16 22:26
• Updated: 08/Dec/16 10:42
• Resolved: 07/Dec/16 20:23

Description

STR: Install cluster with 4 services: HDFS, Flume, AMS and Smartsense, with custom PID's
Metrics Monitor start fails while starting services

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_monitor.py", line 68, in <module>
    AmsMonitor().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 282, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_monitor.py", line 42, in start
    action = 'start'
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
    return fn(*args, **kwargs)
  File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams_service.py", line 103, in ams_service
    user=params.ams_user
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/sbin/ambari-metrics-monitor --config /etc/ambari-metrics-monitor/conf start' returned 1. ######## Hortonworks #############
This is MOTD message, added for testing in qe infra
Building psutil...
running build
running build_py
creating build/lib.linux-x86_64-2.6
creating build/lib.linux-x86_64-2.6/psutil
copying psutil/__init__.py -> build/lib.linux-x86_64-2.6/psutil
copying psutil/_common.py -> build/lib.linux-x86_64-2.6/psutil
copying psutil/_compat.py -> build/lib.linux-x86_64-2.6/psutil
copying psutil/_psbsd.py -> build/lib.linux-x86_64-2.6/psutil
copying psutil/_pslinux.py -> build/lib.linux-x86_64-2.6/psutil
copying psutil/_psosx.py -> build/lib.linux-x86_64-2.6/psutil
copying psutil/_psposix.py -> build/lib.linux-x86_64-2.6/psutil
copying psutil/_pssunos.py -> build/lib.linux-x86_64-2.6/psutil
copying psutil/_pswindows.py -> build/lib.linux-x86_64-2.6/psutil
running build_ext
building '_psutil_linux' extension
creating build/temp.linux-x86_64-2.6
creating build/temp.linux-x86_64-2.6/psutil
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-2.6/psutil/_psutil_linux.o
psutil/_psutil_linux.c: In function ‘psutil_proc_cpu_affinity_set’:
psutil/_psutil_linux.c:327: warning: suggest explicit braces to avoid ambiguous ‘else’
gcc -pthread -shared build/temp.linux-x86_64-2.6/psutil/_psutil_linux.o -L/usr/lib64 -lpython2.6 -o build/lib.linux-x86_64-2.6/_psutil_linux.so
building '_psutil_posix' extension
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c psutil/_psutil_posix.c -o build/temp.linux-x86_64-2.6/psutil/_psutil_posix.o
gcc -pthread -shared build/temp.linux-x86_64-2.6/psutil/_psutil_posix.o -L/usr/lib64 -lpython2.6 -o build/lib.linux-x86_64-2.6/_psutil_posix.so
Verifying Python version compatibility...
Using python  /usr/bin/python2.6
Checking for previously running Metric Monitor...
Starting ambari-metrics-monitor
/usr/sbin/ambari-metrics-monitor: line 63: /grid/0/pid/metric_monitor/ambari-metrics-monitor.pid: Permission denied
ERROR:  Cannot write pid /grid/0/pid/metric_monitor/ambari-metrics-monitor.pid.

Issue Links

• links to: ReviewBoard