Ivy
  1. Ivy
  2. IVY-1148

Encountered 'multiple artifacts retrieved to same file' error when module does not have multiple artifacts

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.2.0-RC1
    • Component/s: Core
    • Labels:
      None
    • Environment:

      Windows or Linux, i386

      Description

      Recently we started experiencing intermittent problems with the error 'multiple artifacts retrieved to same module' for artifacts that did not have multiple artifacts.

      I have not been able to create a simple contrived error, though I can generally reproduce it in our complex set of environments and artifacts.

      It started happening when we started using the branch attribute in our modules. Usually the problem happens when a module has a both a direct and an indirect dependency on another module. We were working around it by excluding the module from being resolved indirectly.

      During debugging, I traced it to the fact that a certain identical module revision was not being handled as identical. Specifically, in line 335 of RetrieveEngine, the artifact was being added to the conflictsReports HashSet. The ArtifactDownloadReport.equals() was determining them to be equal, so I did not expect them to be inserted. But the ArtifactDownloadReport.hashCode() was coming up with different integers.

      The cause of this is that one of the artifacts had an additional qualified attributed called 'merged' which was not different, even though this was the same version of the same artifact. The value of the attribute looked like:
      orgA#moduleA#trunk;build1245 -> orgB#moduleB#trunk;build833

      Where module Z is the module being resolved, and it has a direct dependency on module A and moduleB, and moduleB has a direct dependency on moduleA.

      So because of the different qualified attribute, an artifact that represented the same file was returning a different hash code.

      I'm not sure what this extra 'merged' information represents. It seems to represent something about how the artifact was resolved. There is no possible retrieve pattern that could (or should) differentiate artifacts that differ only in the 'merged' attribute, so I think this is a little too strict.

      My strategy, for which I am attaching a patch that passes existing unit tests, is to use a hashCode() and equals() method that represents the minimum necessary to determine whether an artifact maps to a unique file.

      1. ArtifactDownloadReport.patch
        1 kB
        Carlton Brown
      2. ivy_1148.xml
        0.5 kB
        Mark Harrah

        Activity

        Hide
        Carlton Brown added a comment -

        Proposed solution that passes existing unit tests.

        Show
        Carlton Brown added a comment - Proposed solution that passes existing unit tests.
        Hide
        Maarten Coene added a comment -

        I think I've found a bug in the equals of AbstractArtifact (it doesn't compare the publicationDate properly), maybe this is causing your problem?

        Maarten

        Show
        Maarten Coene added a comment - I think I've found a bug in the equals of AbstractArtifact (it doesn't compare the publicationDate properly), maybe this is causing your problem? Maarten
        Hide
        Carlton Brown added a comment -

        In my case the publication dates are equal. The false inequality causing the problem is definitely caused by inequality of the 'merged' qualifed extended attribute of AbstractArtifact. It is null for one artifact but not the other, yet these are in fact the same artifact.

        Show
        Carlton Brown added a comment - In my case the publication dates are equal. The false inequality causing the problem is definitely caused by inequality of the 'merged' qualifed extended attribute of AbstractArtifact. It is null for one artifact but not the other, yet these are in fact the same artifact.
        Hide
        Maarten Coene added a comment - - edited

        I've made an update to the class IvyNode which I hope will fix the problem (but it's hard to tell without a junit test).
        Could you please give it a try and post your feedback here?

        EDIT: you might have to clean your ivy cache first

        thanks;
        Maarten

        Show
        Maarten Coene added a comment - - edited I've made an update to the class IvyNode which I hope will fix the problem (but it's hard to tell without a junit test). Could you please give it a try and post your feedback here? EDIT: you might have to clean your ivy cache first thanks; Maarten
        Hide
        Carlton Brown added a comment - - edited

        The test passes in my environment with the latest trunk revision 888191. The failure does not occur anymore. Thanks for working on this with the incomplete information. Can you briefly explain the fix?

        Show
        Carlton Brown added a comment - - edited The test passes in my environment with the latest trunk revision 888191. The failure does not occur anymore. Thanks for working on this with the incomplete information. Can you briefly explain the fix?
        Hide
        Maarten Coene added a comment -

        Thanks for the feedback, it could only be fixed so quickly thanks to your excellent detective/debugging work

        The ivy:merged attributes was introduced to fix IVY-537. That gave me a little understanding about this attribute, so all I did was to make sure that if for some reason such an merged artifact was also selected in a normal way, the merged artifact wasn't returned from IvyNode and so won't get retrieved by the RetrieveEngine.

        Maarten

        Show
        Maarten Coene added a comment - Thanks for the feedback, it could only be fixed so quickly thanks to your excellent detective/debugging work The ivy:merged attributes was introduced to fix IVY-537 . That gave me a little understanding about this attribute, so all I did was to make sure that if for some reason such an merged artifact was also selected in a normal way, the merged artifact wasn't returned from IvyNode and so won't get retrieved by the RetrieveEngine. Maarten
        Hide
        Mark Harrah added a comment -

        If you need a simple example for a unit test, the attached Ivy file works with trunk but not 2.1.0. In 2.1.0, it gives the error:

        Multiple artifacts of the module commons-beanutils#commons-beanutils;1.8.2 are retrieved to the same file!

        when using ivy:retrieve.

        Thanks,
        Mark

        Show
        Mark Harrah added a comment - If you need a simple example for a unit test, the attached Ivy file works with trunk but not 2.1.0. In 2.1.0, it gives the error: Multiple artifacts of the module commons-beanutils#commons-beanutils;1.8.2 are retrieved to the same file! when using ivy:retrieve. Thanks, Mark
        Hide
        Peter Bogdanowicz added a comment -

        I am using version 2.2.0-RC1 and until now everything was fine. No i have started using the pattern attribute in my retrieve target. This is where the problem started. Now i always get "Multiple artifacts of the module commons-lang#commons-lang;2.4 are retrieved to the same file! Update the retrieve pattern to fix this error.". Could it be that the bug was only fixed partially? Thx!

        Show
        Peter Bogdanowicz added a comment - I am using version 2.2.0-RC1 and until now everything was fine. No i have started using the pattern attribute in my retrieve target. This is where the problem started. Now i always get "Multiple artifacts of the module commons-lang#commons-lang;2.4 are retrieved to the same file! Update the retrieve pattern to fix this error.". Could it be that the bug was only fixed partially? Thx!
        Hide
        Carlton Brown added a comment -

        The bug wasn't to make that error go away forever, it was to ensure that the error is only reported when the retrieve pattern truly has a problem.

        Since you didn't post your retrieve pattern, I can only guess at the problem, but the usual mistake is assuming that modules only publish library jars. If you're resolving a module like commons-lang that publishes other jars (source jars, javadoc jars, etc) then you need to incorporate the [classifier] token in your retrieve pattern. Here's an example: pattern="$

        {lib.dir}

        /[conf]/[artifact](-[classifier]).[ext]"

        Show
        Carlton Brown added a comment - The bug wasn't to make that error go away forever, it was to ensure that the error is only reported when the retrieve pattern truly has a problem. Since you didn't post your retrieve pattern, I can only guess at the problem, but the usual mistake is assuming that modules only publish library jars. If you're resolving a module like commons-lang that publishes other jars (source jars, javadoc jars, etc) then you need to incorporate the [classifier] token in your retrieve pattern. Here's an example: pattern="$ {lib.dir} / [conf] / [artifact] (- [classifier] ). [ext] "

          People

          • Assignee:
            Maarten Coene
            Reporter:
            Carlton Brown
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development