[HADOOP-18883] Expect-100 JDK bug resolution: prevent multiple server calls

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0, 3.3.9, 3.5.0, 3.4.1
• Component/s: fs/azure
• Labels: pull-request-available
• Target Version/s: 3.5.0
• Assignee: Pranav Saxena
• Reporter: Pranav Saxena
• Created: 07/Sep/23 11:28
• Updated: 11/Feb/24 01:10
• Resolved: 21/Jan/24 19:15

Description

This is inline to JDK bug: https://bugs.openjdk.org/browse/JDK-8314978.

 
With the current implementation of HttpURLConnection if server rejects the “Expect 100-continue” then there will be ‘java.net.ProtocolException’ will be thrown from 'expect100Continue()' method.

After the exception thrown, If we call any other method on the same instance (ex getHeaderField(), or getHeaderFields()). They will internally call getOuputStream() which invokes writeRequests(), which make the actual server call. 

In the AbfsHttpOperation, after sendRequest() we call processResponse() method from AbfsRestOperation. Even if the conn.getOutputStream() fails due to expect-100 error, we consume the exception and let the code go ahead. So, we can have getHeaderField() / getHeaderFields() / getHeaderFieldLong() which will be triggered after getOutputStream is failed. These invocation will lead to server calls.

Issue Links

• relates to: HADOOP-18865 ABFS: Adding 100 continue in userAgent String and dynamically removing it if retry is without the header enabled.
• links to: GitHub Pull Request #6022
• links to: GitHub Pull Request #6484
• links to: GitHub Pull Request #6511
• links to: https://github.com/apache/hadoop/pull/6022