[HADOOP-10289] o.a.h.u.ReflectionUtils.printThreadInfo() causes deadlock in TestHttpServer

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.3.0, 3.0.0-alpha1
• Component/s: util

Environment

MacOS X 10.9/Java 6 1.6.0_65-b14-462

• Assignee: Unassigned
• Reporter: Laurent Goujon
• Created: 25/Jan/14 20:05
• Updated: 12/May/16 21:06

Description

This bug is a followup on HADOOP-9964

ReflectionUtils.printThreadInfo is now a synchronized method. This change creates sometimes deadlock situation in TestHttpServer if one servlet thread calling this method is waiting on client to consume output.

In TestHttpServer, several tests connect to the http server only to check the status code but without reading the full inputstream. Depending on HttpURLConnection, the deadlock scenario may be triggered or not.

Note that in the original ticket, it is not explained why synchronized fixed the issue. According to the attached stacktrace, test was blocked on HttpServer.stop(), waiting on worker threads to stop, which didn't happen because those threads were waiting for their output to be consumed, so the original issue looks very similar to what I'm experiencing.

My proposed fix is to remove synchronized (as it seems to make the issue worse) but configure HttpServer.stop() to forcibly kill threads after a configurable period of time

Issue Links

• relates to: HADOOP-9964 O.A.H.U.ReflectionUtils.printThreadInfo() is not thread-safe which cause TestHttpServer pending 10 minutes or longer.