[OAK-5005] S3 async upload fails to catch and log exceptions other than DataStoreException

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.21, 1.4.10, 1.5.13, 1.6.0
• Component/s: blob
• Assignee: Amit Jain
• Reporter: Amit Jain
• Created: 26/Oct/16 06:50
• Updated: 14/Nov/16 11:35
• Resolved: 26/Oct/16 08:08

Description

The code AsyncUploadJob#run only catches and logs DataStoreException but any exceptions due to n/w connectivity etc. like below are ignored.
Any exceptions should be caught and clearly logged, mentioning the identifier and file which failed. It should also add to the retry queue or mechanism already used.

21.10.2016 16:44:52.354 *INFO* [s3-write-worker-58] com.amazonaws.http.AmazonHttpClient Unable to execute HTTP request: hostname-bucket.s3-eu-west-1.amazonaws.com: unknown error
java.net.UnknownHostException: hostname-bucket.s3-eu-west-1.amazonaws.com: unknown error
	at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
	at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
	at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
	at java.net.InetAddress.getAllByName(InetAddress.java:1192)
	at java.net.InetAddress.getAllByName(InetAddress.java:1126)
	at com.amazonaws.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:27)
	at com.amazonaws.http.DelegatingDnsResolver.resolve(DelegatingDnsResolver.java:38)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.resolveHostname(DefaultClientConnectionOperator.java:259)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:159)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
	at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:749)
	at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:505)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:317)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3595)
	at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:999)
	at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:977)
	at org.apache.jackrabbit.oak.blob.cloud.aws.s3.S3Backend.write(S3Backend.java:773)
	at org.apache.jackrabbit.oak.blob.cloud.aws.s3.S3Backend.access$600(S3Backend.java:72)
	at org.apache.jackrabbit.oak.blob.cloud.aws.s3.S3Backend$AsyncUploadJob.run(S3Backend.java:1066)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)