[HIVE-20040] JDBC: HTTP listen queue is 50 and SYNs are lost

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0, 3.0.0
• Fix Version/s: 3.2.0
• Component/s: JDBC
• Labels: Branch3Candidate
• Target Version/s: 3.2.0
• Assignee: Gopal Vijayaraghavan
• Reporter: Gopal Vijayaraghavan
• Created: 30/Jun/18 22:12
• Updated: 31/Jul/18 21:13
• Resolved: 31/Jul/18 21:13

Description

When testing with 5000 concurrent users, the JDBC HTTP port ends up overflowing on SYNs when the HS2 gc pauses.

This is because each getQueryProgress request is an independent HTTP request, so unlike the BINARY mode, there are more connections being established & closed in HTTP mode.

LISTEN     0      50           *:10004                    *:* 

This turns into connection errors when enabling net.ipv4.tcp_abort_on_overflow=1, but the better approach is to enqueue the connections until the HS2 is done with its GC pause.