# Maven doesn't try to download a dependency when it already exists a version with classifier locally

## Details

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 2.0.9, 2.0.10, 2.1.0
• Fix Version/s:
• Component/s:
• Labels:
None
• Environment:
Java 5, Windows XP

## Description

When maven installs in the local repository an artifact with a classifier, and not the principal artifact, it won't try in a reactor to download the principal artifact from the remote repository.
I created a testcase to reproduce it. It's quite simple. You unzip it and in the root directory you launch

mvn site

This build uses clover, thus it installs locally cloverified versions of artifacts.
The build will fail because it doesn't find the SNAPSHOT of the module1 :

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

Missing:
----------

Then, install it using the command:
mvn install:install-file -DgroupId=org.apache.maven.it.test-metadata-local-clover -DartifactId=module1 -Dversion=1.0.0-SNAPSHOT -Dpackaging=jar -Dfile=/pa
th/to/file

Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=org.apache.maven.it.test-metadata-local-clover -DartifactId=module1 -Dversion=1.0.0-SNAPSHOT -Dpackaging=jar -Dfile=/path
/to/file -Durl=[url] -DrepositoryId=[id]

Path to dependency:

----------
1 required artifact is missing.

for artifact:

from the specified remote repositories:
central (http://maven-proxy.groupe.generali.fr/all),


It's anormal because I set a "local" remote repository in the build where it should try to download it.
You can validate that it is working by launching :

mvn install -f module2/pom.xml

This bug is really annoying for us. It exists for a long long time now. I thought it was due to a problem with metadata sent by archiva, but after migrating to nexus the problem always occurs.
I don't know if the problem is in metadata or in the reactor itself. I think it is the second one. There are many issues opened about the reactor and classifiers.
Is there some who can try if it can reproduce the error with my testcase ?
It should be easy to create a real IT for maven with it if it is necessary.

## Attachments

1. MNG-4142.patch
2 kB
David Rousselie
21 kB
Arnaud HERITIER

## Activity

Hide
Arnaud HERITIER added a comment -

Something else to add : If I launch after the site build, another build which doesn't produce the main artefact (for example : mvn sonar), it will also fail trying to download dependency. I think that it can be related to maven-metadata and more probably to the maven maven-metadata-local.xml which is generated.
After the mvn site, I have in my local repository in org\apache\maven\it\test-metadata-local-clover\module1\1.0.0-SNAPSHOT:

maven-metadata-central.xml
module1-1.0.0-SNAPSHOT-clover.jar
module1-1.0.0-SNAPSHOT.pom


and in each metadata file :

<?xml version="1.0" encoding="UTF-8"?>
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<artifactId>module1</artifactId>
<version>1.0.0-SNAPSHOT</version>

<?xml version="1.0" encoding="UTF-8"?>
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<artifactId>module1</artifactId>
<version>1.0.0-SNAPSHOT</version>
<versioning>
<snapshot>
<localCopy>true</localCopy>
</snapshot>
<lastUpdated>20090423144418</lastUpdated>
</versioning>

<?xml version="1.0" encoding="UTF-8"?>
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<artifactId>module1</artifactId>
<version>1.0.0-SNAPSHOT</version>
<versioning>
<snapshot>
<timestamp>20090423.140624</timestamp>
<buildNumber>1</buildNumber>
</snapshot>
<lastUpdated>20090423140625</lastUpdated>
</versioning>


Do you think that maven-metadata-local.xml is good ? It says that there's a local copy whereas there's no module1-1.0.0-SNAPSHOT.jar in the directory.

Show
Hide
David Rousselie added a comment -

The problem is that when the Maven resolve the artifact version with maven-metadata-local.xml when it exists and it resolves it to 1.0.0-SNAPSHOT whereas only module1-1.0.0-SNAPSHOT-clover.jar (in this exemple). Then when it try to access the file artifact (module1-1.0.0-SNAPSHOT.jar) and fails, it tries to get the same version on remote repositories whereas it should have try to resolve the artifact version with data found in maven-metadata-${remotereponame} .xml before trying to get it from each repository. I have attached a small patch (base on revision 805295 in branch maven-2.2.x) which might not be the best way to correct this bug : while resolving artifact version, it loads local metadata only if artifact's file exists locally. Show David Rousselie added a comment - The problem is that when the Maven resolve the artifact version with maven-metadata-local.xml when it exists and it resolves it to 1.0.0-SNAPSHOT whereas only module1-1.0.0-SNAPSHOT-clover.jar (in this exemple). Then when it try to access the file artifact (module1-1.0.0-SNAPSHOT.jar) and fails, it tries to get the same version on remote repositories whereas it should have try to resolve the artifact version with data found in maven-metadata-$ {remotereponame} .xml before trying to get it from each repository. I have attached a small patch (base on revision 805295 in branch maven-2.2.x) which might not be the best way to correct this bug : while resolving artifact version, it loads local metadata only if artifact's file exists locally.
Hide
David Rousselie added a comment -

A possible correction for bug MNG-4142

Show
David Rousselie added a comment - A possible correction for bug MNG-4142
Hide
Peter Liljenberg added a comment -

I've encountered this problem as well when doing distributed builds (using Hudson).
When one build server (#1) builds and deploys an artifact the maven-metadata-local.xml will contain the localCopy attribute:
<versioning>
<snapshot>
<localCopy>true</localCopy>
</snapshot>
<lastUpdated>20091127124227</lastUpdated>
</versioning>

Which means that if another build server builds the same module AFTER server #1 and produce a new artifact in the repository, machine #1 will never download this new SNAPSHOT version when trying to build artifacts depending on the artifact.
This is a huge problem (and a scary one!) for us and I would like to see a fix for this soon

Show
Peter Liljenberg added a comment - I've encountered this problem as well when doing distributed builds (using Hudson). When one build server (#1) builds and deploys an artifact the maven-metadata-local.xml will contain the localCopy attribute: <versioning> <snapshot> <localCopy>true</localCopy> </snapshot> <lastUpdated>20091127124227</lastUpdated> </versioning> Which means that if another build server builds the same module AFTER server #1 and produce a new artifact in the repository, machine #1 will never download this new SNAPSHOT version when trying to build artifacts depending on the artifact. This is a huge problem (and a scary one!) for us and I would like to see a fix for this soon
Hide
Brian Laframboise added a comment - - edited

I just ran into this issue myself (with Maven 2.2.1). A build on one of my build machines was failing due to maven not downloading a newer version of a dependency that was available on Nexus because it had previously installed an earlier version of that same dependency into the local repository.

When I removed the <localCopy>true</localCopy> element from the metadata of the non-updating dependency the problem went away - and dutifully came back when I re-added it.

Peter, I agree - this is a scary problem for anybody trying to implement a distributed build system. What I don't understand is why more people are not encountering this issue.

For now I'll attempt a workaround this by adding a cron job, per build agent, to frequently change <localCopy> back to false. If anybody has a better suggestion, I'd love to hear it.

Show
Brian Laframboise added a comment - - edited I just ran into this issue myself (with Maven 2.2.1). A build on one of my build machines was failing due to maven not downloading a newer version of a dependency that was available on Nexus because it had previously installed an earlier version of that same dependency into the local repository. When I removed the <localCopy>true</localCopy> element from the metadata of the non-updating dependency the problem went away - and dutifully came back when I re-added it. Peter, I agree - this is a scary problem for anybody trying to implement a distributed build system. What I don't understand is why more people are not encountering this issue. For now I'll attempt a workaround this by adding a cron job, per build agent, to frequently change <localCopy> back to false. If anybody has a better suggestion, I'd love to hear it.
Hide
Martin Todorov added a comment -

Hi,

I would like to ask why the attached patch is not being accepted?
There are currently over three dozen votes for this and it is apparent that people are hitting the problem.

In addition, perhaps the title of the bug is not quite correct. This doesn't just happen for artifacts with classifiers. In our case it happens for POM files where there are several inherited POMs.
We're using Hudson and Nexus had to limit the parents to build on one Hudson node, while all modules which depend on them - on the rest of the nodes.

I also think this is a huge and very obscure problem (esp. with distributed builds) and, provided that there is already a patch, I believe it's about time for it to be reviewed.

Regards,

Martin

Show
Martin Todorov added a comment - Hi, I would like to ask why the attached patch is not being accepted? There are currently over three dozen votes for this and it is apparent that people are hitting the problem. In addition, perhaps the title of the bug is not quite correct. This doesn't just happen for artifacts with classifiers. In our case it happens for POM files where there are several inherited POMs. We're using Hudson and Nexus had to limit the parents to build on one Hudson node, while all modules which depend on them - on the rest of the nodes. I also think this is a huge and very obscure problem (esp. with distributed builds) and, provided that there is already a patch, I believe it's about time for it to be reviewed. Regards, Martin
Hide
Arnaud HERITIER added a comment -

Were you able to test if the problem exists with maven 3 ?
I agree that this issue is really annoying but for now there is no 2.X version release scheduled.
That's why there was no feedback from dev team.

Show
Arnaud HERITIER added a comment - Were you able to test if the problem exists with maven 3 ? I agree that this issue is really annoying but for now there is no 2.X version release scheduled. That's why there was no feedback from dev team.
Hide
Martin Todorov added a comment -

Bonjour Arnaud!

As per your request, I have tested this with Maven 3.0 today. I am afraid the problem still persists. Today this actually happened with a completely different project/setup/machine under both Maven 2.2.1 and 3.0.

This is not just annoying, it's actually a show-stopper when it happens to a developer who's not expecting it, as they spend quite some time trying to figure it out.

Please review for the release of Maven 3.1, (if a 2.x version will not be further released).

Regards,

Martin

Show
Martin Todorov added a comment - Bonjour Arnaud! As per your request, I have tested this with Maven 3.0 today. I am afraid the problem still persists. Today this actually happened with a completely different project/setup/machine under both Maven 2.2.1 and 3.0. This is not just annoying, it's actually a show-stopper when it happens to a developer who's not expecting it, as they spend quite some time trying to figure it out. Please review for the release of Maven 3.1, (if a 2.x version will not be further released). Regards, Martin
Hide

I've the same problem but with standard SNAPSHOT dependencies, not using classifier, and with 3.0.3.

Show
nodje added a comment - I've the same problem but with standard SNAPSHOT dependencies, not using classifier, and with 3.0.3.
Hide
Martin Todorov added a comment - - edited

This bug/feature doesn't really have much to do with classifiers. I believe the original bug report filer just thought it had something to do with the problem.
Perhaps somebody should rename the bug to "Maven doesn't try to download SNAPSHOT artifacts, if you've installed them locally; even if you pass -U".

I believe that by using -U the local metadata should get "unlocked" (which currently doesn't do this) for the locally installed SNAPSHOT artifacts.

Show
Martin Todorov added a comment - - edited This bug/feature doesn't really have much to do with classifiers. I believe the original bug report filer just thought it had something to do with the problem. Perhaps somebody should rename the bug to "Maven doesn't try to download SNAPSHOT artifacts, if you've installed them locally; even if you pass -U". I believe that by using -U the local metadata should get "unlocked" (which currently doesn't do this) for the locally installed SNAPSHOT artifacts.
Hide

alright, good then, that's my problem!

Can't wait for the fix to come, hopefully as early as 3.0.4...

Show
nodje added a comment - alright, good then, that's my problem! Can't wait for the fix to come, hopefully as early as 3.0.4...
Hide
nodje added a comment - - edited

We're a year after the latest comment, could someone update the Fix version, that'd give some hope.

Our team is severely being affected by this problem, and it seems to be larger than the description suggests.
We have SNAPSHOT update problems on a daily basis, even for deps than haven't been installed locally.
A CI server deploy the artifact, and in some case no one seem to get the updated artifact.

Thanks for pointing me to other bug report if my problem is out of scope here.

Show
nodje added a comment - - edited We're a year after the latest comment, could someone update the Fix version, that'd give some hope. Our team is severely being affected by this problem, and it seems to be larger than the description suggests. We have SNAPSHOT update problems on a daily basis, even for deps than haven't been installed locally. A CI server deploy the artifact, and in some case no one seem to get the updated artifact. Thanks for pointing me to other bug report if my problem is out of scope here.
Hide
Martin Todorov added a comment -

I have replied to numerous posts on Stack Overflow concerning this particular issue. People keep hitting this and are unhappy with the current behavior. In the company I work for, we had to make Jenkins use private repositories for each project which would be removed before every build, just for the sake of sanity and for us not losing our minds when a build fails due to this reason.

The problem here is also observed under Maven 3. Furthermore, it's not a classifier problem. It happens to whatever dependencies you have to modules which you've installed locally.

My suggestion would be to make the -U option "unlock" the Maven metadata files so that they are fetched from the remote repository.

I would also appreciate some response from the you guys. What's the problem here that keeps this issue unscheduled for future releases? We haven't received an official comment for a long while. :-|

Show
Martin Todorov added a comment - I have replied to numerous posts on Stack Overflow concerning this particular issue. People keep hitting this and are unhappy with the current behavior. In the company I work for, we had to make Jenkins use private repositories for each project which would be removed before every build, just for the sake of sanity and for us not losing our minds when a build fails due to this reason. The problem here is also observed under Maven 3. Furthermore, it's not a classifier problem. It happens to whatever dependencies you have to modules which you've installed locally. My suggestion would be to make the -U option "unlock" the Maven metadata files so that they are fetched from the remote repository. I would also appreciate some response from the you guys. What's the problem here that keeps this issue unscheduled for future releases? We haven't received an official comment for a long while. :-|
Hide

Removing a private repository before each build would indeed solve my problem
I'm considering the option, since I'm losing my mind with all the build failing due to SNAPSHOT update problem.

Developers new to Maven in our team don't even understand "SNAPSHOT update", they just think they have to install manually each SNAPSHOT dependency... which messes up even more the whole thing.

And I insist, our CI server build agents don't install SNAPSHOT locally, and still they fail to retrieve the latest snapshots in many cases.

It's not a locally installed artifact update problem ONLY.

Show
nodje added a comment - Removing a private repository before each build would indeed solve my problem I'm considering the option, since I'm losing my mind with all the build failing due to SNAPSHOT update problem. Developers new to Maven in our team don't even understand "SNAPSHOT update", they just think they have to install manually each SNAPSHOT dependency... which messes up even more the whole thing. And I insist, our CI server build agents don't install SNAPSHOT locally, and still they fail to retrieve the latest snapshots in many cases. It's not a locally installed artifact update problem ONLY.
Hide
Cody Fyler added a comment -

We are experiencing this issue with Maven 3.0.5 on Jenkins with multiple slaves.

It's kind of a nightmare.

Can we please get this at least assigned?

Here is the workaround that I came up with that in my opinion really sucks, but it works.

http://maven.apache.org/plugins/maven-dependency-plugin/purge-local-repository-mojo.html#include

In this case, I use a filter to clear out the SNAPSHOT artifacts that need to be updated. But it sucks that I have to do it. And we have a LOT of projects where we are going to have to do something similar.

Show
Cody Fyler added a comment - We are experiencing this issue with Maven 3.0.5 on Jenkins with multiple slaves. It's kind of a nightmare. Can we please get this at least assigned? Here is the workaround that I came up with that in my opinion really sucks, but it works. http://maven.apache.org/plugins/maven-dependency-plugin/purge-local-repository-mojo.html#include In this case, I use a filter to clear out the SNAPSHOT artifacts that need to be updated. But it sucks that I have to do it. And we have a LOT of projects where we are going to have to do something similar.
Hide

Good luck, you just have to live with it or spend the time to understand the Maven sources by yourself.
See Martin Todorov comments for a work around.

Show
nodje added a comment - Good luck, you just have to live with it or spend the time to understand the Maven sources by yourself. See Martin Todorov comments for a work around.
Hide
John Chatham added a comment -

Ran into this issue recently on maven 3.0.4; our workaround was to update to maven 3.2.1, where there's a new property one can set: maven.install.skip

Set that to true, either with -Dmaven.install.skip or (how we did it) by adding a property entry in the maven settings file our jenkins build was using, and that fixed that; we build and deploy our snapshots, but don't install them into the local repositories.

Show
John Chatham added a comment - Ran into this issue recently on maven 3.0.4; our workaround was to update to maven 3.2.1, where there's a new property one can set: maven.install.skip Set that to true, either with -Dmaven.install.skip or (how we did it) by adding a property entry in the maven settings file our jenkins build was using, and that fixed that; we build and deploy our snapshots, but don't install them into the local repositories.

## People

• Assignee:
Unassigned
Reporter:
Arnaud HERITIER