[AMBARI-17510] LogSearch REST Integration component can cause performance issues

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.4.0
• Component/s: ambari-server
• Assignee: Bob Nettleton
• Reporter: Bob Nettleton
• Created: 30/Jun/16 20:27
• Updated: 01/Jul/16 20:11
• Resolved: 01/Jul/16 18:11

Description

In larger clusters, or clusters that are long-running, some performance-related problems have been uncovered that occur when LogSearch is deployed in an Ambari cluster.

In particular, if the LogSearch Server itself is performing slowly, this can cause a degradation of service in the Ambari REST API, since the LogSearch integration components make direct REST calls to obtain the Logging data on behalf of the Ambari UI.

This problem is caused by two issues:

1. The REST requests occur on the Ambari REST request thread. This can be problematic if a given request hangs. This can cause the Ambari REST API, and the Ambari server itself, to become unresponsive.

2. The Ambari REST integration layer makes HTTP calls to the LogSearch service to obtain Logging metadata, that is then attached to the HostComponent resource in Ambari. These HTTP calls are not currently configured with a connect or read timeout, which can cause a failing call to wait indefinitely, which again can cause Ambari to become unresponsive.

I'm working on a fix for this, and will be submitting a patch shortly.

Issue Links

• links to: ReviewBoard