Uploaded image for project: 'Maven Assembly Plugin'
  1. Maven Assembly Plugin
  2. MASSEMBLY-848

Version range dependencies not resolved from the reactor

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Description

      When using modules that have independent version numbers (that is, modules in the same project may have different version numbers), it's commonplace to specify dependencies between modules with version ranges when using semantic versioning.

      For some reason, when version ranges are used on dependencies that refer to modules that are part of the project (and therefore should be in the reactor), the assembly plugin ignores them and tries to resolve them from the local repository instead.

      The following project reproduces this issue (just "mvn clean package"):

      https://github.com/io7m/independent-versioning-20170207

      Interestingly, this didn't happen with the same assembly plugin on older Maven versions. Here's a successful build on Travis CI:

      https://travis-ci.org/io7m/independent-versioning-20170207

      1. mvn.log.gz
        25 kB
        Mark Raynsford

        Issue Links

          Activity

          Hide
          io7m Mark Raynsford added a comment -

          Compressed output of mvn -X clean package

          Show
          io7m Mark Raynsford added a comment - Compressed output of mvn -X clean package
          Hide
          io7m Mark Raynsford added a comment -

          This is also reproduceable with the current tip of the master branch of Maven (66a0fbe495f7df45562bee22188341899e1ca839).

          Show
          io7m Mark Raynsford added a comment - This is also reproduceable with the current tip of the master branch of Maven (66a0fbe495f7df45562bee22188341899e1ca839).
          Hide
          io7m Mark Raynsford added a comment -

          The problem is here (this entire method, really):

          https://github.com/apache/maven-plugins/blob/trunk/maven-assembly-plugin/src/main/java/org/apache/maven/plugins/assembly/artifact/DefaultDependencyResolver.java#L232

          I'm debugging based on the contents of an integration test, the
          original source files of which came from this example project:

          https://github.com/io7m/independent-versioning-20170207

          When line 232 is reached in the debugger, the dependencyArtifacts
          parameter contains the two artifacts
          com.io7m.experimental:mod-a:jar:1.1.0:compile and
          com.io7m.experimental:mod-b:jar:1.2.0:compile and both of those claim
          to already have been resolved (isResolved() == true), which makes sense
          as they're reactor projects. Both are present in the given
          configSource.reactorProjects list. However, the code doesn't appear to
          make any use of this list or the fact that the artifacts are already
          resolved. It simply passes the list of artifacts to the resolver which
          then fails with an OverConstrainedVersionException because the reactor
          project artifacts aren't in any of the remote or local repositories.

          This seems to be the root cause, but I can't work out
          whether this is expected behaviour, misuse of an API, an oversight, or
          possibly all or none of the above!

          Show
          io7m Mark Raynsford added a comment - The problem is here (this entire method, really): https://github.com/apache/maven-plugins/blob/trunk/maven-assembly-plugin/src/main/java/org/apache/maven/plugins/assembly/artifact/DefaultDependencyResolver.java#L232 I'm debugging based on the contents of an integration test, the original source files of which came from this example project: https://github.com/io7m/independent-versioning-20170207 When line 232 is reached in the debugger, the dependencyArtifacts parameter contains the two artifacts com.io7m.experimental:mod-a:jar:1.1.0:compile and com.io7m.experimental:mod-b:jar:1.2.0:compile and both of those claim to already have been resolved (isResolved() == true), which makes sense as they're reactor projects. Both are present in the given configSource.reactorProjects list. However, the code doesn't appear to make any use of this list or the fact that the artifacts are already resolved. It simply passes the list of artifacts to the resolver which then fails with an OverConstrainedVersionException because the reactor project artifacts aren't in any of the remote or local repositories. This seems to be the root cause, but I can't work out whether this is expected behaviour, misuse of an API, an oversight, or possibly all or none of the above!
          Hide
          io7m Mark Raynsford added a comment -

          Even worse: If a previous version of these artifacts have been installed and the current version range includes them, the old artifacts in the local repository will be used in preference to the reactor project artifacts.

          Show
          io7m Mark Raynsford added a comment - Even worse: If a previous version of these artifacts have been installed and the current version range includes them, the old artifacts in the local repository will be used in preference to the reactor project artifacts.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user io7m opened a pull request:

          https://github.com/apache/maven-plugins/pull/104

          MASSEMBLY-848 Do not attempt to resolve reactor projects

          I'm submitting this pull request in the hope of getting some discussion on the issue. I feel like the submitted fix here is not the right way to solve this problem but do not understand enough of the way Maven resolves artifacts to be sure. This change does at least appear to resolve the provided integration test properly and does not break any existing tests.

          ```
          This is an attempt at addressing MASSEMBLY-848: When resolving the
          dependencies of a project, we explicitly avoid attempting to resolve
          any dependency artifacts that are also reactor projects.
          ```

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/io7m/maven-plugins trunk

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/maven-plugins/pull/104.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #104


          commit c1c9576c7cedc3cfb06d79350251f2a4296b308c
          Author: Mark Raynsford <code@io7m.com>
          Date: 2017-02-22T20:36:09Z

          MASSEMBLY-848 Do not attempt to resolve reactor projects

          This is an attempt at addressing MASSEMBLY-848: When resolving the
          dependencies of a project, we explicitly avoid attempting to resolve
          any dependency artifacts that are also reactor projects.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user io7m opened a pull request: https://github.com/apache/maven-plugins/pull/104 MASSEMBLY-848 Do not attempt to resolve reactor projects I'm submitting this pull request in the hope of getting some discussion on the issue. I feel like the submitted fix here is not the right way to solve this problem but do not understand enough of the way Maven resolves artifacts to be sure. This change does at least appear to resolve the provided integration test properly and does not break any existing tests. ``` This is an attempt at addressing MASSEMBLY-848 : When resolving the dependencies of a project, we explicitly avoid attempting to resolve any dependency artifacts that are also reactor projects. ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/io7m/maven-plugins trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/maven-plugins/pull/104.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #104 commit c1c9576c7cedc3cfb06d79350251f2a4296b308c Author: Mark Raynsford <code@io7m.com> Date: 2017-02-22T20:36:09Z MASSEMBLY-848 Do not attempt to resolve reactor projects This is an attempt at addressing MASSEMBLY-848 : When resolving the dependencies of a project, we explicitly avoid attempting to resolve any dependency artifacts that are also reactor projects.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user io7m closed the pull request at:

          https://github.com/apache/maven-plugins/pull/104

          Show
          githubbot ASF GitHub Bot added a comment - Github user io7m closed the pull request at: https://github.com/apache/maven-plugins/pull/104
          Hide
          io7m Mark Raynsford added a comment -

          No, I'm sorry, I can't fix this. Something seems to be very fundamentally broken about how version ranges are handled and I have no idea how to fix it. Unless someone else steps up, this one is going to stay broken (which essentially makes the Assembly plugin useless for our purposes).

          Show
          io7m Mark Raynsford added a comment - No, I'm sorry, I can't fix this. Something seems to be very fundamentally broken about how version ranges are handled and I have no idea how to fix it. Unless someone else steps up, this one is going to stay broken (which essentially makes the Assembly plugin useless for our purposes).
          Hide
          khmarbaise Karl Heinz Marbaise added a comment - - edited

          Using the following in your assembly descriptor works:

          <assembly
            xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
          
            <id>distribution</id>
            <baseDirectory>${project.name}-${project.version}</baseDirectory>
          
            <formats>
              <format>zip</format>
            </formats>
          
            <moduleSets>
              <moduleSet>
                <useAllReactorProjects>true</useAllReactorProjects>
                <binaries>
                  <includeDependencies>false</includeDependencies>
                  <outputDirectory>lib</outputDirectory>
                  <unpack>false</unpack>
                  <dependencySets>
                    <dependencySet>
                      <useProjectArtifact>false</useProjectArtifact>
                      <useTransitiveDependencies>false</useTransitiveDependencies>
                    </dependencySet>
                  </dependencySets>
                </binaries>
              </moduleSet>
            </moduleSets>
          </assembly>
          

          Added working example.

          Show
          khmarbaise Karl Heinz Marbaise added a comment - - edited Using the following in your assembly descriptor works: <assembly xmlns= "http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation= "http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd" > <id> distribution </id> <baseDirectory> ${project.name}-${project.version} </baseDirectory> <formats> <format> zip </format> </formats> <moduleSets> <moduleSet> <useAllReactorProjects> true </useAllReactorProjects> <binaries> <includeDependencies> false </includeDependencies> <outputDirectory> lib </outputDirectory> <unpack> false </unpack> <dependencySets> <dependencySet> <useProjectArtifact> false </useProjectArtifact> <useTransitiveDependencies> false </useTransitiveDependencies> </dependencySet> </dependencySets> </binaries> </moduleSet> </moduleSets> </assembly> Added working example .
          Hide
          io7m Mark Raynsford added a comment -

          Interesting, thank you. I'll try it shortly.

          I don't really understand why this now necessary on newer Maven versions. Did something fundamental change inside Maven that somehow changed the semantics of the Assembly plugin?

          Show
          io7m Mark Raynsford added a comment - Interesting, thank you. I'll try it shortly. I don't really understand why this now necessary on newer Maven versions. Did something fundamental change inside Maven that somehow changed the semantics of the Assembly plugin?
          Hide
          khmarbaise Karl Heinz Marbaise added a comment -

          The original example does not work on Maven 3.0.5, 3.1.1, 3.2.5, 3.3.9 and 3.5.0. It is independent of the Maven version. The same output on all Maven versions. The problem is related to a wrong assumption, cause a dependencySet is something different than a moduleSet. In this case it is related to the moduleSet. And at the moment I don't see that there is an issue...

          Show
          khmarbaise Karl Heinz Marbaise added a comment - The original example does not work on Maven 3.0.5, 3.1.1, 3.2.5, 3.3.9 and 3.5.0. It is independent of the Maven version. The same output on all Maven versions. The problem is related to a wrong assumption, cause a dependencySet is something different than a moduleSet. In this case it is related to the moduleSet. And at the moment I don't see that there is an issue...
          Hide
          io7m Mark Raynsford added a comment -

          > The original example does not work on Maven 3.0.5, 3.1.1, 3.2.5, 3.3.9 and 3.5.0

          https://s3.amazonaws.com/archive.travis-ci.org/jobs/199286373/log.txt

          ^ That's a log of it working on 3.2.5 (or perhaps failing to fail!).

          Show
          io7m Mark Raynsford added a comment - > The original example does not work on Maven 3.0.5, 3.1.1, 3.2.5, 3.3.9 and 3.5.0 https://s3.amazonaws.com/archive.travis-ci.org/jobs/199286373/log.txt ^ That's a log of it working on 3.2.5 (or perhaps failing to fail!).
          Hide
          khmarbaise Karl Heinz Marbaise added a comment -

          I have taken the above link to github and cloned that and tested that with each Maven version which failed (I used mvn clean package)...The point here is that you have used mvn install instead of mvn package which is something different.

          If you do a mvn install each module of that multi module build will be iterated over. The first one will do the life cycle including installing the artifact into local cache $HOME/.m2/repository the next module etc. and so until the last module mod-c which has the maven-assembly-plugin configuration which now resolves the dependencies from the local cache which works. This will work with all Maven versions cause it's independent of the Maven version.

          If you like to do mvn clean package that will fail for all Maven versions cause they are not solved from the reactor only if artifacts have the same version this will work...

          I never use mvn clean install on multi module builds. I always use mvn clean package...and that's the issue here. The resolution will not work for those dependencies from the reactor cause they have different version numbers and they will be tried to resolve from remote repository in which they did not exist...If you use the example I have shown this will work also with mvn clean package...

          Show
          khmarbaise Karl Heinz Marbaise added a comment - I have taken the above link to github and cloned that and tested that with each Maven version which failed (I used mvn clean package )...The point here is that you have used mvn install instead of mvn package which is something different. If you do a mvn install each module of that multi module build will be iterated over. The first one will do the life cycle including installing the artifact into local cache $HOME/.m2/repository the next module etc. and so until the last module mod-c which has the maven-assembly-plugin configuration which now resolves the dependencies from the local cache which works. This will work with all Maven versions cause it's independent of the Maven version. If you like to do mvn clean package that will fail for all Maven versions cause they are not solved from the reactor only if artifacts have the same version this will work... I never use mvn clean install on multi module builds. I always use mvn clean package ...and that's the issue here. The resolution will not work for those dependencies from the reactor cause they have different version numbers and they will be tried to resolve from remote repository in which they did not exist...If you use the example I have shown this will work also with mvn clean package ...
          Hide
          io7m Mark Raynsford added a comment -

          Understood. Thank you!

          Show
          io7m Mark Raynsford added a comment - Understood. Thank you!
          Hide
          io7m Mark Raynsford added a comment -

          Actually, this still seems to be a problem... I posted the above example as a simple reproduction case, but it's actually a much simplified version of a real project I was using as an experiment in independent module versioning:

          https://github.com/io7m/thoth

          The problem now is that because I have to use `moduleSets` to create a distribution zip containing the project jars and all dependencies, and because I have to use `<useTransitiveDependencies>false</useTransitiveDependencies>` in order to prevent Maven from trying to pick up the project's module jars from the local repos, the dependencies of each module are excluded... For example, the `com.io7m.thoth.command.system` module depends on `com.io7m.junreachable.core`, but `com.io7m.junreachable.core` is not included in the final distribution zip because transitive dependencies aren't followed.

          Is there some better way to do what I'm trying to do?

          Show
          io7m Mark Raynsford added a comment - Actually, this still seems to be a problem... I posted the above example as a simple reproduction case, but it's actually a much simplified version of a real project I was using as an experiment in independent module versioning: https://github.com/io7m/thoth The problem now is that because I have to use `moduleSets` to create a distribution zip containing the project jars and all dependencies, and because I have to use `<useTransitiveDependencies>false</useTransitiveDependencies>` in order to prevent Maven from trying to pick up the project's module jars from the local repos, the dependencies of each module are excluded... For example, the `com.io7m.thoth.command.system` module depends on `com.io7m.junreachable.core`, but `com.io7m.junreachable.core` is not included in the final distribution zip because transitive dependencies aren't followed. Is there some better way to do what I'm trying to do?

            People

            • Assignee:
              Unassigned
              Reporter:
              io7m Mark Raynsford
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development