Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3180

Impala Daemon Ready Status leading to Monitor-HostMonitor throttling_logger ERROR

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Invalid
    • Affects Version/s: Impala 2.2
    • Fix Version/s: None
    • Component/s: Infrastructure
    • Labels:

      Description

      When I check from cloudera-scm-agent logs.

      [11/Mar/2016 10:50:19 +0000] 17000 Metadata-Plugin navigator_plugin_pipeline INFO Stopping Navigator Plugin Pipeline '' for cluster-host-inspector (log dir: None)
      [11/Mar/2016 10:50:23 +0000] 17000 CP Server Thread-10 _cplogging INFO 10.81.80.34 - - [11/Mar/2016:10:50:23] "GET /process/9408-cluster-host-inspector/files/inspector HTTP/1.1" 200 2425 "" "Java/1.7.0_67"
      [11/Mar/2016 10:50:38 +0000] 17000 MainThread agent INFO Process with same id has changed: 9408-cluster-host-inspector.
      [11/Mar/2016 10:50:38 +0000] 17000 MainThread agent INFO Deactivating process 9408-cluster-host-inspector
      [11/Mar/2016 10:50:39 +0000] 17000 Metadata-Plugin navigator_plugin INFO stopping Metadata Plugin for cluster-host-inspector with pipelines []
      [11/Mar/2016 10:50:39 +0000] 17000 Metadata-Plugin navigator_plugin_pipeline INFO Stopping Navigator Plugin Pipeline '' for cluster-host-inspector (log dir: None)
      [11/Mar/2016 10:50:41 +0000] 17000 Audit-Plugin navigator_plugin INFO stopping Audit Plugin for cluster-host-inspector with pipelines []
      [11/Mar/2016 10:50:41 +0000] 17000 Audit-Plugin navigator_plugin_pipeline INFO Stopping Navigator Plugin Pipeline '' for cluster-host-inspector (log dir: None)
      [11/Mar/2016 10:52:33 +0000] 17000 Monitor-HostMonitor throttling_logger ERROR Kill subprocess exception with args ['/usr/java/jdk1.7.0_67-cloudera/bin/java', '-classpath', '/usr/share/cmf/lib/agent-5.4.1.jar', 'com.cloudera.cmon.agent.DnsTest']
      Traceback (most recent call last):
      File "/usr/lib64/cmf/agent/src/cmf/subprocess_timeout.py", line 84, in subprocess_with_timeout
      os.kill(p.pid, signal.SIGTERM)
      OSError: [Errno 3] No such process
      [11/Mar/2016 10:52:33 +0000] 17000 Monitor-HostMonitor throttling_logger ERROR (1 skipped) Timeout with args ['/usr/java/jdk1.7.0_67-cloudera/bin/java', '-classpath', '/usr/share/cmf/lib/agent-5.4.1.jar', 'com.cloudera.cmon.agent.DnsTest']
      Traceback (most recent call last):
      File "/usr/lib64/cmf/agent/src/cmf/subprocess_timeout.py", line 84, in subprocess_with_timeout
      os.kill(p.pid, signal.SIGTERM)
      OSError: [Errno 3] No such process
      [11/Mar/2016 10:52:33 +0000] 17000 Monitor-HostMonitor throttling_logger ERROR (1 skipped) Failed to collect java-based DNS names
      Traceback (most recent call last):
      File "/usr/lib64/cmf/agent/src/cmf/monitor/host/dns_names.py", line 64, in collect
      result, stdout, stderr = self._subprocess_with_timeout(args, self._poll_timeout)
      File "/usr/lib64/cmf/agent/src/cmf/monitor/host/dns_names.py", line 46, in _subprocess_with_timeout
      return subprocess_with_timeout(args, timeout)
      File "/usr/lib64/cmf/agent/src/cmf/subprocess_timeout.py", line 94, in subprocess_with_timeout
      raise Exception("timeout with args %s" % args)
      Exception: timeout with args ['/usr/java/jdk1.7.0_67-cloudera/bin/java', '-classpath', '/usr/share/cmf/lib/agent-5.4.1.jar', 'com.cloudera.cmon.agent.DnsTest']
      [11/Mar/2016 10:52:52 +0000] 17000 ImpalaDaemonQueryMonitoring throttling_logger ERROR (54 skipped) Error fetching executing query ids at 'http://dcslpd43.amat.com:25000/inflight_query_ids'
      Traceback (most recent call last):
      File "/usr/lib64/cmf/agent/src/cmf/monitor/impalad/query_monitor.py", line 479, in get_executing_query_ids
      password=password)
      File "/usr/lib64/cmf/agent/src/cmf/url_util.py", line 62, in urlopen_with_timeout
      return opener.open(url, data, timeout)
      File "/usr/lib64/python2.6/urllib2.py", line 391, in open
      response = self._open(req, data)
      File "/usr/lib64/python2.6/urllib2.py", line 409, in _open
      '_open', req)
      File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
      result = func(*args)
      File "/usr/lib64/python2.6/urllib2.py", line 1190, in http_open
      return self.do_open(httplib.HTTPConnection, req)
      File "/usr/lib64/python2.6/urllib2.py", line 1165, in do_open
      raise URLError(err)
      URLError: <urlopen error timed out>

      could some one please assist on this.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              manthinechaitu_impala_39b9 chaitanya
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: