Maven
  1. Maven
  2. MNG-757

Transitive dependency resolution ignores custom repositories

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0-beta-1
    • Fix Version/s: 2.0-beta-1
    • Component/s: None
    • Labels:
      None
    • Environment:
      Windows XP, Cygwin

      Description

      The attached files set the scene:

      • test-repo.zip - expand this into the root context of a local web server on 127.0.0.1:8080 for a test repo
      • projects.zip - expand this for the projects
      • settings.xml - the ~/.m2/settings.xml file

      The scenario is as follows:

      • test-repo contains a single artifact C
      • project B depends on C
      • project A depends on B & defines test-repo
      • settings.xml also defines test-repo

      The build process is:

      • m2 install B (downloads C, installs B ok)
      • m2 install A (finds B and C in local repo, installs A ok)

      Now the problem is if C is then deleted the second step fails - i.e. m2 only looks in the central repo for C and not the custom test-repo, even though test-repo is defined in both A's POM and settings.xml.

      This didn't happen in 2.0-alpha-3 - is this intentional?

      1. test-repo.zip
        2 kB
        Mark Hobson
      2. settings.xml
        0.3 kB
        Mark Hobson
      3. projects.zip
        0.8 kB
        Mark Hobson

        Activity

        Mark Hobson created issue -
        Hide
        Brett Porter added a comment -

        yeah, this is a bug. probably a regression related to recent fixes in profile merging.

        Show
        Brett Porter added a comment - yeah, this is a bug. probably a regression related to recent fixes in profile merging.
        Brett Porter made changes -
        Field Original Value New Value
        Fix Version/s 2.0-beta-1 [ 11040 ]
        Assignee John Casey [ jdcasey ]
        Hide
        Brett Porter added a comment -

        btw, this is easier to test using a file:// repo than a test repo on http

        Show
        Brett Porter added a comment - btw, this is easier to test using a file:// repo than a test repo on http
        Hide
        John Casey added a comment -

        this would seem to imply that we expect dependency poms to have profiles from settings.xml merged into them...that doesn't sound right to me. The only way that a could transitively find c would be if:

        1. the same set of repositories were used at all levels for dependency resolution...which would be defined in the project-to-be-built pom.xml file. This is how -alpha-3 worked.

        2. we apply settings.xml profiles to all POMs we work with during the build, including dependency POMs.

        If you think about the integrity of the dependency POM, these two are restatements of the same thing...modifying something that's meant to be an authoritative source of information for a graph of dependencies (the dependency and it's transitive closure).

        I'm not sure it's wise to put all resolution control in the hands of your project's dependencies...so maybe it would be better to say that the project being built should provide the list of repositories used to resolve everything a la -alpha-3...? I bring this up since there was another JIRA issue filed to change that to repository-set-per-POM.

        Show
        John Casey added a comment - this would seem to imply that we expect dependency poms to have profiles from settings.xml merged into them...that doesn't sound right to me. The only way that a could transitively find c would be if: 1. the same set of repositories were used at all levels for dependency resolution...which would be defined in the project-to-be-built pom.xml file. This is how -alpha-3 worked. 2. we apply settings.xml profiles to all POMs we work with during the build, including dependency POMs. If you think about the integrity of the dependency POM, these two are restatements of the same thing...modifying something that's meant to be an authoritative source of information for a graph of dependencies (the dependency and it's transitive closure). I'm not sure it's wise to put all resolution control in the hands of your project's dependencies...so maybe it would be better to say that the project being built should provide the list of repositories used to resolve everything a la -alpha-3...? I bring this up since there was another JIRA issue filed to change that to repository-set-per-POM.
        John Casey made changes -
        Original Estimate 4h [ 14400 ]
        Complexity Intermediate Expert
        Remaining Estimate 4h [ 14400 ]
        Hide
        Brett Porter added a comment -

        I think settings profiles should be applied to all poms - parents, deps and projects. The reason for this need to is to avoid the chicken and egg problem of a parent in a repository specified in the parent (which may happen in the project, or the dep - so needs to apply to both).

        It may make sense to make the project authoratative, but that doesn't solve the chicken and egg problem either.

        The other issue you refer to was to make the project repositories be searched before the parent (ie super POM), I think.

        Show
        Brett Porter added a comment - I think settings profiles should be applied to all poms - parents, deps and projects. The reason for this need to is to avoid the chicken and egg problem of a parent in a repository specified in the parent (which may happen in the project, or the dep - so needs to apply to both). It may make sense to make the project authoratative, but that doesn't solve the chicken and egg problem either. The other issue you refer to was to make the project repositories be searched before the parent (ie super POM), I think.
        Hide
        Mark Hobson added a comment -

        Not sure if the actual use-case that led to this helps, but I had:

        1) project depending on Hibernate
        2) Hibernate depending on JTA
        3) JTA hosted on my repo

        So JTA on my repo couldn't possibly be referenced by the ibiblio Hibernate POM, and so m2 needs help finding it. Either this could come from settings.xml, or maybe even a repo-hint added to the Hibernate dependency in the top-level project's POM?

        I haven't been keeping up-to-speed on the latest design discussions, so whatever you guys think as long as there's a way of achieving this.

        Show
        Mark Hobson added a comment - Not sure if the actual use-case that led to this helps, but I had: 1) project depending on Hibernate 2) Hibernate depending on JTA 3) JTA hosted on my repo So JTA on my repo couldn't possibly be referenced by the ibiblio Hibernate POM, and so m2 needs help finding it. Either this could come from settings.xml, or maybe even a repo-hint added to the Hibernate dependency in the top-level project's POM? I haven't been keeping up-to-speed on the latest design discussions, so whatever you guys think as long as there's a way of achieving this.
        Hide
        John Casey added a comment -

        I think the correct solution here is to append successively deep transitive dependency POMs' repository definitions to the list being used for resolution, preserving repository uniqueness by id. This allows each project to have precedence in determining which repository a given artifact will be resolved from, while ensuring that the artifact will be resolved from somewhere...as long as those repositories are accessible.

        Show
        John Casey added a comment - I think the correct solution here is to append successively deep transitive dependency POMs' repository definitions to the list being used for resolution, preserving repository uniqueness by id. This allows each project to have precedence in determining which repository a given artifact will be resolved from, while ensuring that the artifact will be resolved from somewhere...as long as those repositories are accessible.
        Hide
        Mark Hobson added a comment -

        Sounds good to me. Would higher-level POMs have precedence over transitive dependency POMs? This would allow projects to override repositories, e.g. use an in-house repo for some dependencies rather than ibiblio (without mirroring the entirety of ibiblio).

        Show
        Mark Hobson added a comment - Sounds good to me. Would higher-level POMs have precedence over transitive dependency POMs? This would allow projects to override repositories, e.g. use an in-house repo for some dependencies rather than ibiblio (without mirroring the entirety of ibiblio).
        John Casey made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        John Casey logged work - 22/Aug/05 15:30
        • Time Spent:
          2h
           
          Simply aggregating remote repositories down the transitivity line...local project repositories have precedence over those of the dependencies, and so on. This ensures that locally defined repositories can host artifacts that would otherwise be resolved from the central repository, for example. Super-POM repositories will only be appended to the end, to ensure that any other repository has the first chance to resolve it.
        John Casey made changes -
        Time Spent 2h [ 7200 ]
        Remaining Estimate 4h [ 14400 ] 0h [ 0 ]
        John Casey made changes -
        Resolution Fixed [ 1 ]
        Status In Progress [ 3 ] Closed [ 6 ]
        Brett Porter made changes -
        Resolution Fixed [ 1 ]
        Status Closed [ 6 ] Reopened [ 4 ]
        Hide
        Brett Porter added a comment -

        rolled back due to MNG-836.

        Show
        Brett Porter added a comment - rolled back due to MNG-836 .
        Brett Porter made changes -
        Fix Version/s 2.0-beta-2 [ 11861 ]
        Fix Version/s 2.0-beta-1 [ 11040 ]
        Hide
        John Casey added a comment -

        what are the details of the regression caused by this fix, do we know? Do we have a test in place to prevent future regressions?

        Show
        John Casey added a comment - what are the details of the regression caused by this fix, do we know? Do we have a test in place to prevent future regressions?
        Hide
        Brett Porter added a comment -

        the details are in MNG-836

        Show
        Brett Porter added a comment - the details are in MNG-836
        Hide
        John Casey added a comment -

        I'm functionally re-adding the code that got rolled back, because I suspect that this was not the root problem. In addition, I'm adding two integration tests: the first, it0068, will verify that MNG-836 is not repeated; the second, it2001, replicates the scenario originally laid out in this issue, and verifies that it works. This last test - it2001 - is complex, and must be run manually by executing test.sh inside the it2001 directory.

        I'm going to commit what I have for this, and if regressions pop up, I guess we'll fix them.

        Show
        John Casey added a comment - I'm functionally re-adding the code that got rolled back, because I suspect that this was not the root problem. In addition, I'm adding two integration tests: the first, it0068, will verify that MNG-836 is not repeated; the second, it2001, replicates the scenario originally laid out in this issue, and verifies that it works. This last test - it2001 - is complex, and must be run manually by executing test.sh inside the it2001 directory. I'm going to commit what I have for this, and if regressions pop up, I guess we'll fix them.
        John Casey made changes -
        Status Reopened [ 4 ] In Progress [ 3 ]
        Hide
        John Casey added a comment -

        I've worked another 3h on this issue, but I'm not at all sure how to log it properly without causing negative time estimation in JIRA...

        Show
        John Casey added a comment - I've worked another 3h on this issue, but I'm not at all sure how to log it properly without causing negative time estimation in JIRA...
        Hide
        John Casey added a comment -

        see it0068 and it2001

        Show
        John Casey added a comment - see it0068 and it2001
        John Casey made changes -
        Fix Version/s 2.0-beta-2 [ 11861 ]
        Resolution Fixed [ 1 ]
        Status In Progress [ 3 ] Closed [ 6 ]
        Fix Version/s 2.0-beta-1 [ 11040 ]
        Vincent Massol made changes -
        Workflow Maven [ 38809 ] Maven New [ 47886 ]
        Mark Thomas made changes -
        Project Import Sun Apr 05 08:49:45 UTC 2015 [ 1428223785911 ]
        Mark Thomas made changes -
        Workflow jira [ 12712048 ] Default workflow, editable Closed status [ 12751428 ]
        Mark Thomas made changes -
        Project Import Sun Apr 05 21:45:26 UTC 2015 [ 1428270326204 ]
        Mark Thomas made changes -
        Workflow jira [ 12948870 ] Default workflow, editable Closed status [ 12988157 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        3d 3h 29m 1 John Casey 22/Aug/05 15:02
        Closed Closed Reopened Reopened
        14d 4h 4m 1 Brett Porter 05/Sep/05 19:36
        Reopened Reopened In Progress In Progress
        8d 2h 45m 1 John Casey 13/Sep/05 22:21
        In Progress In Progress Closed Closed
        56m 42s 2 John Casey 13/Sep/05 22:48

          People

          • Assignee:
            John Casey
            Reporter:
            Mark Hobson
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 4h
              4h
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 2h Time Not Required
              2h

                Development