[HADOOP-13044] Amazon S3 library 10.10.60+ (JDK8u60+) depends on http components 4.3

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.8.0
• Component/s: build, fs/s3

Environment

JDK 8u60

• Target Version/s: 2.8.0
• Assignee: Kai
• Reporter: Kai
• Created: 21/Apr/16 05:43
• Updated: 19/May/16 01:19
• Resolved: 18/May/16 19:22

Description

In case of using AWS SDK in the classpath of hadoop, we faced an issue caused by incompatiblity of AWS SDK and httpcomponents.

java.lang.NoSuchFieldError: INSTANCE
	at com.amazonaws.http.conn.SdkConnectionKeepAliveStrategy.getKeepAliveDuration(SdkConnectionKeepAliveStrategy.java:48)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:535)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)

The latest AWS SDK depends on 4.3.x which has DefaultConnectionKeepAliveStrategy.INSTANCE. This field is introduced from 4.3.

This will allow us to avoid CLASSPATH confliction around httpclient versions.

Issue Links

• depends upon: HADOOP-13050 Upgrade to AWS SDK 1.11.45
• is part of: HADOOP-12767 update apache httpclient version to 4.5.2; httpcore to 4.4.4
• is related to: HADOOP-12767 update apache httpclient version to 4.5.2; httpcore to 4.4.4