Minor Description
I found that ivy download time was very long especially when downloading from maven. While investigating that, I noticed for each artifact download was done 3 times. I could see that by checking the logs from the download done from my thanks to the facts that some artifacts are downloaded from one of my servers. See the following logs to illustrate this:
81.188.70.167 - - [17/Jan/2006:13:04:23 -0500] "GET /ivy-repository/jetty/org.mortbay.jetty-5.1.8.jar HTTP/1.1" 200 676610 "-" "Java/1.5.0_06"
81.188.70.167 - - [17/Jan/2006:13:04:23 -0500] "GET /ivy-repository/jetty/org.mortbay.jetty-5.1.8.jar HTTP/1.1" 200 676610 "-" "Java/1.5.0_06"
81.188.70.167 - - [17/Jan/2006:13:04:23 -0500] "GET /ivy-repository/jetty/org.mortbay.jetty-5.1.8.jar HTTP/1.1" 200 676610 "-" "Java/1.5.0_06"
I then investigated the ivy sources (v 1.2a) to find out that this is due to the way BasicURLHandler is implemented. Indeed, each artifact is downloaded using 3 different requests:
- to check the artifact is existing
- to find the artifact size
- to do the actual download.
Not very efficient is it?
Using commons-http-client improves a little bit by reducing the requests to 1 HEAD and 2 GETs.
I've verified that using commons-http-client and a little bit refactoring, it can be reduced to something more ideal: 1 HEAD, 1 GET.
The problem comes from the way the URLHandler.isReachable is used by the URLResource class. When an artifact exists it uses URLConnection just to obtain its size. I modified isReachable to return the size of the artifact. The implementation of the HEAD method in commons-http-client can obtain the size without doing a real download.
Now I have a more efficient usage of the server, this is shown in the following log:
81.188.70.167 - - [17/Jan/2006:14:13:53 -0500] "HEAD /ivy-repository/hessian/hessian-3.0.13.jar HTTP/1.1" 200 - "-" "Jakarta Commons-HttpClient/3.0"
81.188.70.167 - - [17/Jan/2006:14:13:54 -0500] "GET /ivy-repository/hessian/hessian-3.0.13.jar HTTP/1.1" 200 187171 "-" "Jakarta Commons-HttpClient/3.0"
I will post a patch to this issue.