[WHIRR-349] Retry if blobstore put fails

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.6.0
• Fix Version/s: 0.6.0
• Component/s: core

Environment

Ubuntu 11.04
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

(using https://github.com/bdumon/whirr 13cd70e which uses a jclouds 1.1.0-SNAPSHOT)

• Assignee: Adrian Cole (Inactive)
• Reporter: Karel Vervaeke
• Created: 29/Jul/11 08:05
• Updated: 14/Aug/11 08:57
• Resolved: 14/Aug/11 08:57

Description

When the blobstore fails to upload a file, it might be worth retrying a couple of times rather than just aborting
the whole process.
OTOH, the failure would occur before any machines are launched, so the loss is limited.

Here's an example stacktrace of a failed upload:
Created blob cache container 'fr6mxqa4d6aa' located in '[id=EU, scope=REGION, description=EU, parent=aws-s3, iso3166Codes=[IE], metadata={}]'
Uploading 'mydecorator-1.0-SNAPSHOT.tar.gz' to 'fr6mxqa4d6aa' blob cache.
Jul 29, 2011 8:53:50 AM org.jclouds.logging.jdk.JDKLogger logWarn
WARNING: Cannot retry after server error, command is not replayable: [method=AWSS3AsyncClient.putObject, request=PUT https://fr6mxqa4d6aa.s3.amazonaws.com/mydecorator-1.0-SNAPSHOT.tar.gz HTTP/1.1]
Exception in thread "main" org.jclouds.http.HttpResponseException: error after writing 335872/637480 bytes to PUT https://fr6mxqa4d6aa.s3.amazonaws.com/mydecorator-1.0-SNAPSHOT.tar.gz HTTP/1.1 connecting to PUT https://fr6mxqa4d6aa.s3.amazonaws.com/mydecorator-1.0-SNAPSHOT.tar.gz HTTP/1.1
at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:183)
at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:134)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.RuntimeException: error after writing 335872/637480 bytes to PUT https://fr6mxqa4d6aa.s3.amazonaws.com/mydecorator-1.0-SNAPSHOT.tar.gz HTTP/1.1
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:245)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:77)
at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:156)
... 6 more
Caused by: java.io.IOException: Error writing request body to server
at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(HttpURLConnection.java:2716)
at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:2699)
at com.google.common.io.CountingOutputStream.write(CountingOutputStream.java:51)
at com.google.common.io.ByteStreams.copy(ByteStreams.java:172)
at org.jclouds.io.payloads.BasePayload.writeTo(BasePayload.java:67)
at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:243)
... 8 more
Removing blob cache 'fr6mxqa4d6aa'

Issue Links

• is blocked by: WHIRR-361 refactor jclouds dependencies
• relates to: WHIRR-359 Document known limitations for Whirr 0.6.0