Maven
  1. Maven
  2. MNG-757

Transitive dependency resolution ignores custom repositories

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0-beta-1
    • Fix Version/s: 2.0-beta-1
    • Component/s: None
    • Labels:
      None
    • Environment:
      Windows XP, Cygwin

      Description

      The attached files set the scene:

      • test-repo.zip - expand this into the root context of a local web server on 127.0.0.1:8080 for a test repo
      • projects.zip - expand this for the projects
      • settings.xml - the ~/.m2/settings.xml file

      The scenario is as follows:

      • test-repo contains a single artifact C
      • project B depends on C
      • project A depends on B & defines test-repo
      • settings.xml also defines test-repo

      The build process is:

      • m2 install B (downloads C, installs B ok)
      • m2 install A (finds B and C in local repo, installs A ok)

      Now the problem is if C is then deleted the second step fails - i.e. m2 only looks in the central repo for C and not the custom test-repo, even though test-repo is defined in both A's POM and settings.xml.

      This didn't happen in 2.0-alpha-3 - is this intentional?

      1. projects.zip
        0.8 kB
        Mark Hobson
      2. settings.xml
        0.3 kB
        Mark Hobson
      3. test-repo.zip
        2 kB
        Mark Hobson

        Activity

        Hide
        Brett Porter added a comment -

        yeah, this is a bug. probably a regression related to recent fixes in profile merging.

        Show
        Brett Porter added a comment - yeah, this is a bug. probably a regression related to recent fixes in profile merging.
        Hide
        Brett Porter added a comment -

        btw, this is easier to test using a file:// repo than a test repo on http

        Show
        Brett Porter added a comment - btw, this is easier to test using a file:// repo than a test repo on http
        Hide
        John Casey added a comment -

        this would seem to imply that we expect dependency poms to have profiles from settings.xml merged into them...that doesn't sound right to me. The only way that a could transitively find c would be if:

        1. the same set of repositories were used at all levels for dependency resolution...which would be defined in the project-to-be-built pom.xml file. This is how -alpha-3 worked.

        2. we apply settings.xml profiles to all POMs we work with during the build, including dependency POMs.

        If you think about the integrity of the dependency POM, these two are restatements of the same thing...modifying something that's meant to be an authoritative source of information for a graph of dependencies (the dependency and it's transitive closure).

        I'm not sure it's wise to put all resolution control in the hands of your project's dependencies...so maybe it would be better to say that the project being built should provide the list of repositories used to resolve everything a la -alpha-3...? I bring this up since there was another JIRA issue filed to change that to repository-set-per-POM.

        Show
        John Casey added a comment - this would seem to imply that we expect dependency poms to have profiles from settings.xml merged into them...that doesn't sound right to me. The only way that a could transitively find c would be if: 1. the same set of repositories were used at all levels for dependency resolution...which would be defined in the project-to-be-built pom.xml file. This is how -alpha-3 worked. 2. we apply settings.xml profiles to all POMs we work with during the build, including dependency POMs. If you think about the integrity of the dependency POM, these two are restatements of the same thing...modifying something that's meant to be an authoritative source of information for a graph of dependencies (the dependency and it's transitive closure). I'm not sure it's wise to put all resolution control in the hands of your project's dependencies...so maybe it would be better to say that the project being built should provide the list of repositories used to resolve everything a la -alpha-3...? I bring this up since there was another JIRA issue filed to change that to repository-set-per-POM.
        Hide
        Brett Porter added a comment -

        I think settings profiles should be applied to all poms - parents, deps and projects. The reason for this need to is to avoid the chicken and egg problem of a parent in a repository specified in the parent (which may happen in the project, or the dep - so needs to apply to both).

        It may make sense to make the project authoratative, but that doesn't solve the chicken and egg problem either.

        The other issue you refer to was to make the project repositories be searched before the parent (ie super POM), I think.

        Show
        Brett Porter added a comment - I think settings profiles should be applied to all poms - parents, deps and projects. The reason for this need to is to avoid the chicken and egg problem of a parent in a repository specified in the parent (which may happen in the project, or the dep - so needs to apply to both). It may make sense to make the project authoratative, but that doesn't solve the chicken and egg problem either. The other issue you refer to was to make the project repositories be searched before the parent (ie super POM), I think.
        Hide
        Mark Hobson added a comment -

        Not sure if the actual use-case that led to this helps, but I had:

        1) project depending on Hibernate
        2) Hibernate depending on JTA
        3) JTA hosted on my repo

        So JTA on my repo couldn't possibly be referenced by the ibiblio Hibernate POM, and so m2 needs help finding it. Either this could come from settings.xml, or maybe even a repo-hint added to the Hibernate dependency in the top-level project's POM?

        I haven't been keeping up-to-speed on the latest design discussions, so whatever you guys think as long as there's a way of achieving this.

        Show
        Mark Hobson added a comment - Not sure if the actual use-case that led to this helps, but I had: 1) project depending on Hibernate 2) Hibernate depending on JTA 3) JTA hosted on my repo So JTA on my repo couldn't possibly be referenced by the ibiblio Hibernate POM, and so m2 needs help finding it. Either this could come from settings.xml, or maybe even a repo-hint added to the Hibernate dependency in the top-level project's POM? I haven't been keeping up-to-speed on the latest design discussions, so whatever you guys think as long as there's a way of achieving this.
        Hide
        John Casey added a comment -

        I think the correct solution here is to append successively deep transitive dependency POMs' repository definitions to the list being used for resolution, preserving repository uniqueness by id. This allows each project to have precedence in determining which repository a given artifact will be resolved from, while ensuring that the artifact will be resolved from somewhere...as long as those repositories are accessible.

        Show
        John Casey added a comment - I think the correct solution here is to append successively deep transitive dependency POMs' repository definitions to the list being used for resolution, preserving repository uniqueness by id. This allows each project to have precedence in determining which repository a given artifact will be resolved from, while ensuring that the artifact will be resolved from somewhere...as long as those repositories are accessible.
        Hide
        Mark Hobson added a comment -

        Sounds good to me. Would higher-level POMs have precedence over transitive dependency POMs? This would allow projects to override repositories, e.g. use an in-house repo for some dependencies rather than ibiblio (without mirroring the entirety of ibiblio).

        Show
        Mark Hobson added a comment - Sounds good to me. Would higher-level POMs have precedence over transitive dependency POMs? This would allow projects to override repositories, e.g. use an in-house repo for some dependencies rather than ibiblio (without mirroring the entirety of ibiblio).
        Hide
        Brett Porter added a comment -

        rolled back due to MNG-836.

        Show
        Brett Porter added a comment - rolled back due to MNG-836 .
        Hide
        John Casey added a comment -

        what are the details of the regression caused by this fix, do we know? Do we have a test in place to prevent future regressions?

        Show
        John Casey added a comment - what are the details of the regression caused by this fix, do we know? Do we have a test in place to prevent future regressions?
        Hide
        Brett Porter added a comment -

        the details are in MNG-836

        Show
        Brett Porter added a comment - the details are in MNG-836
        Hide
        John Casey added a comment -

        I'm functionally re-adding the code that got rolled back, because I suspect that this was not the root problem. In addition, I'm adding two integration tests: the first, it0068, will verify that MNG-836 is not repeated; the second, it2001, replicates the scenario originally laid out in this issue, and verifies that it works. This last test - it2001 - is complex, and must be run manually by executing test.sh inside the it2001 directory.

        I'm going to commit what I have for this, and if regressions pop up, I guess we'll fix them.

        Show
        John Casey added a comment - I'm functionally re-adding the code that got rolled back, because I suspect that this was not the root problem. In addition, I'm adding two integration tests: the first, it0068, will verify that MNG-836 is not repeated; the second, it2001, replicates the scenario originally laid out in this issue, and verifies that it works. This last test - it2001 - is complex, and must be run manually by executing test.sh inside the it2001 directory. I'm going to commit what I have for this, and if regressions pop up, I guess we'll fix them.
        Hide
        John Casey added a comment -

        I've worked another 3h on this issue, but I'm not at all sure how to log it properly without causing negative time estimation in JIRA...

        Show
        John Casey added a comment - I've worked another 3h on this issue, but I'm not at all sure how to log it properly without causing negative time estimation in JIRA...
        Hide
        John Casey added a comment -

        see it0068 and it2001

        Show
        John Casey added a comment - see it0068 and it2001

          People

          • Assignee:
            John Casey
            Reporter:
            Mark Hobson
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 4h
              4h
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 2h Time Not Required
              2h

                Development