Uploaded image for project: 'Ivy'
  1. Ivy
  2. IVY-871

test and document PatternVersionMatcher

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.0-RC2
    • Component/s: Core
    • Labels:
      None

      Description

      There is a powerful version matcher in Ivy code base called PatternVersionMatcher. Unfortunately, this matcher is not documented, not tested, and not declared in the default typedef.properties. Nobody remembers where it comes from, but code analysis should be enough to understand how it works and what it can achieve.

      Here's what I understand from a quick analysis:

      settings.xml
      <pattern-vm>
        <match revision="micro" pattern="${major}\.${minor}\.\d+" args="major, minor" matcher="regexp" />
      </pattern-vm>
      
      ivy.xml
      <dependency org="acme" name="foo" rev="micro(1, 3)" />
      

      If I understand correctly the code, this will match any revision like 1.3.<any number>.

      1. patch.txt
        7 kB
        Jon Schneider

        Activity

        Hide
        martinm82 Martin added a comment -

        jaikiran pai, thanks a lot for this perfect explanation.

        Show
        martinm82 Martin added a comment - jaikiran pai , thanks a lot for this perfect explanation.
        Hide
        jaikiran jaikiran pai added a comment -

        So imagine I have the following versions for mylib: 1.2.3, 1.2.4, 1.2.5-17.4, 1.3.0-17.4, 1.3.0

        if I pull now `mylib` with `latestRel(17.4)` it will return `1.3.0-17.4`. The question is only why does it work? I have just defined a regular expression to match this pattern but I do not say anywhere that it should return me the latest one.

        In addition to what I explained in my previous comment, Ivy has the concept of "listing versions from a repository" for a particular dependency if the revision being asked for is a "dynamic" revision. What that means is, in this example:

        java -jar ivy.jar -settings ivysettings.xml -dependency com.acme mylib "latestRel(17.4)"
        

        where your dependency is of the form:

        <dependency org="com.acme" name="mylib" rev="latestRel(17.4)"/>
        

        when the resolution happens, the Ivy resolver implementation checks if the asked revision latestRel(17.4) is a "dynamic revision". Revisions that are backed by a pattern version matcher (like in your case) are considered dynamic revisions. So Ivy resolver implementation, then goes and asks the repository, which is identified for this dependency, to "list the versions" known to that repository, for the dependency module (com.acme:mylib in this case). The repository, in this example, returns the versions 1.2.3, 1.2.4, 1.2.5-17.4, 1.3.0-17.4, 1.3.0.

        The Ivy resolution further has a concept called, "latest strategies"[1], which gets used for dynamic version dependencies, when more than one version is available. When no latest strategy is explicitly configured[2], Ivy uses the default "latest-revision" strategy. The configured (or defaulted) latest strategy is then used to sort the list of versions that were returned by the repository, in the previous step. So in your example, it's going to sort those versions (1.2.3, 1.2.4, 1.2.5-17.4, 1.3.0-17.4, 1.3.0) into a list. This sorted list is then iterated over, by the Ivy resolver implementation and sent to the applicable version matcher (in this case your pattern matcher). The first version that is "accepted" by the matcher is considered as the resolved version for that dependency. So in this example, your pattern matcher gets the 1.3.0-17.4 version befor it gets the 1.2.5-17.4, for evaluation, which it evaluates as "acceptable" (i.e. the matches the pattern of your interest) and hence the resolver ends up using this version.

        [1] https://ant.apache.org/ivy/history/latest-milestone/concept.html#latest
        [2] https://ant.apache.org/ivy/history/latest-milestone/settings/latest-strategies.html

        Show
        jaikiran jaikiran pai added a comment - So imagine I have the following versions for mylib: 1.2.3, 1.2.4, 1.2.5-17.4, 1.3.0-17.4, 1.3.0 if I pull now `mylib` with `latestRel(17.4)` it will return `1.3.0-17.4`. The question is only why does it work? I have just defined a regular expression to match this pattern but I do not say anywhere that it should return me the latest one. In addition to what I explained in my previous comment, Ivy has the concept of "listing versions from a repository" for a particular dependency if the revision being asked for is a "dynamic" revision. What that means is, in this example: java -jar ivy.jar -settings ivysettings.xml -dependency com.acme mylib "latestRel(17.4)" where your dependency is of the form: <dependency org= "com.acme" name= "mylib" rev= "latestRel(17.4)" /> when the resolution happens, the Ivy resolver implementation checks if the asked revision latestRel(17.4) is a "dynamic revision". Revisions that are backed by a pattern version matcher (like in your case) are considered dynamic revisions. So Ivy resolver implementation, then goes and asks the repository, which is identified for this dependency, to "list the versions" known to that repository, for the dependency module (com.acme:mylib in this case). The repository, in this example, returns the versions 1.2.3, 1.2.4, 1.2.5-17.4, 1.3.0-17.4, 1.3.0. The Ivy resolution further has a concept called, "latest strategies"[1], which gets used for dynamic version dependencies, when more than one version is available. When no latest strategy is explicitly configured[2], Ivy uses the default "latest-revision" strategy. The configured (or defaulted) latest strategy is then used to sort the list of versions that were returned by the repository, in the previous step. So in your example, it's going to sort those versions (1.2.3, 1.2.4, 1.2.5-17.4, 1.3.0-17.4, 1.3.0) into a list. This sorted list is then iterated over, by the Ivy resolver implementation and sent to the applicable version matcher (in this case your pattern matcher). The first version that is "accepted" by the matcher is considered as the resolved version for that dependency. So in this example, your pattern matcher gets the 1.3.0-17.4 version befor it gets the 1.2.5-17.4 , for evaluation, which it evaluates as "acceptable" (i.e. the matches the pattern of your interest) and hence the resolver ends up using this version. [1] https://ant.apache.org/ivy/history/latest-milestone/concept.html#latest [2] https://ant.apache.org/ivy/history/latest-milestone/settings/latest-strategies.html
        Hide
        jaikiran jaikiran pai added a comment -

        Sorry, I missed that. I'll take a look and respond soon.

        Show
        jaikiran jaikiran pai added a comment - Sorry, I missed that. I'll take a look and respond soon.
        Hide
        martinm82 Martin added a comment -

        Hi jaikiran pai, did you had a chance to look at my previous comment?

        Show
        martinm82 Martin added a comment - Hi jaikiran pai , did you had a chance to look at my previous comment?
        Hide
        martinm82 Martin added a comment -

        jaikiran pai, thx for your detailed explanation.

        But one thing that I do not understand is why in my defined version matcher is returning the latest revision.

        So imagine I have the following versions for mylib: 1.2.3, 1.2.4, 1.2.5-17.4, 1.3.0-17.4, 1.3.0

        if I pull now `mylib` with `latestRel(17.4)` it will return `1.3.0-17.4`. The question is only why does it work? I have just defined a regular expression to match this pattern but I do not say anywhere that it should return me the latest one.

        Show
        martinm82 Martin added a comment - jaikiran pai , thx for your detailed explanation. But one thing that I do not understand is why in my defined version matcher is returning the latest revision. So imagine I have the following versions for mylib: 1.2.3, 1.2.4, 1.2.5-17.4, 1.3.0-17.4, 1.3.0 if I pull now `mylib` with `latestRel(17.4)` it will return `1.3.0-17.4`. The question is only why does it work? I have just defined a regular expression to match this pattern but I do not say anywhere that it should return me the latest one.
        Hide
        jaikiran jaikiran pai added a comment - - edited

        Martin, thank you for adding those details. I have to admit, the documentation around this matcher needs a bit more work to make it clear what it does. I had to go back and check the code to understand what it's doing.

        In short, what it does is (using your snippet as an example), when it sees that there's a matcher configured for something called latestRel, during dependency resolution, it checks if the revision being asked for, via a dependency matches the revision attribute of the matcher configured. If it does, then it uses that configured matcher to see if it can resolve a version for this dependency. While doing so, the matcher can (optionally) be passed "arguments", a list of which is configured on the matcher through the args attribute. So in your case, you are expecting release as an argument to the matcher. The dependency (in your ivy.xml) passes a value for this argument. In your case, you are passing 17.4 as an argument to that matcher. The matcher can be configured (as you have done in your case) to use the (substituted value) of the argument(s) in the pattern attribute. So in your case, this now translates to the pattern attribute value being \d+\.\d+\.\d+-17.4\.*.

        This value of the pattern attribute is then used for matching/resolving the versions for the dependency. How that pattern value is semantically interpreted is decided by the matcher attribute and in this case you have configured it to be a regex. So your pattern value \d+\.\d+\.\d+-17.4\.* will now be used as a regular expression while evaluating the dependency version latestRel(17.4) asked for as an ivy dependency. I hope that makes it clear on what it does. I'll add this (with some changes maybe) to our documentation this week.

        Show
        jaikiran jaikiran pai added a comment - - edited Martin , thank you for adding those details. I have to admit, the documentation around this matcher needs a bit more work to make it clear what it does. I had to go back and check the code to understand what it's doing. In short, what it does is (using your snippet as an example), when it sees that there's a matcher configured for something called latestRel , during dependency resolution, it checks if the revision being asked for, via a dependency matches the revision attribute of the matcher configured. If it does, then it uses that configured matcher to see if it can resolve a version for this dependency. While doing so, the matcher can (optionally) be passed "arguments", a list of which is configured on the matcher through the args attribute. So in your case, you are expecting release as an argument to the matcher. The dependency (in your ivy.xml) passes a value for this argument. In your case, you are passing 17.4 as an argument to that matcher. The matcher can be configured (as you have done in your case) to use the (substituted value) of the argument(s) in the pattern attribute. So in your case, this now translates to the pattern attribute value being \d+\.\d+\.\d+-17.4\.* . This value of the pattern attribute is then used for matching/resolving the versions for the dependency. How that pattern value is semantically interpreted is decided by the matcher attribute and in this case you have configured it to be a regex . So your pattern value \d+\.\d+\.\d+-17.4\.* will now be used as a regular expression while evaluating the dependency version latestRel(17.4) asked for as an ivy dependency. I hope that makes it clear on what it does. I'll add this (with some changes maybe) to our documentation this week.
        Hide
        martinm82 Martin added a comment - - edited

        Sure. I am publishing artifacts by following semantic versioning plus some extension to mark a released artifact of a specific release. The format is like this 1.2.3-X.Y or 1.2.4-X.Y-SNAPSHOT. All these artifacts are published to the same release repository. So we end up with having various release artifacts (1.2.3-X.Y|-SNAPSHOT) and as well regular ones from trunk (1.2.3).

        In order to get the latest artifacts from a specific release (X.Y) we cannot use the latest.release/snapshot matchers. I read in the documentation about the version matchers and tried to implement something that will get the latest revision from a specific release.

        I have the following ivy settings:

        <match revision="latestRel" pattern="\d+\.\d+\.\d+-${release}\.*" args="release" matcher="regexp" />
        

        I use the Ivy standalone tool by passing the ivysettings.xml and the dependency as following:

        java -jar ivy.jar -settings ivysettings.xml -dependency com.acme mylib "latestRel(17.4)"
        

        This works and pulls only the latest artifact with the release 17.4 only but I do not understand why exactly and have the feeling that it works by accident.

        Show
        martinm82 Martin added a comment - - edited Sure. I am publishing artifacts by following semantic versioning plus some extension to mark a released artifact of a specific release. The format is like this 1.2.3-X.Y or 1.2.4-X.Y-SNAPSHOT . All these artifacts are published to the same release repository. So we end up with having various release artifacts (1.2.3-X.Y|-SNAPSHOT) and as well regular ones from trunk (1.2.3). In order to get the latest artifacts from a specific release (X.Y) we cannot use the latest.release/snapshot matchers. I read in the documentation about the version matchers and tried to implement something that will get the latest revision from a specific release. I have the following ivy settings: <match revision= "latestRel" pattern= "\d+\.\d+\.\d+-${release}\.*" args= "release" matcher= "regexp" /> I use the Ivy standalone tool by passing the ivysettings.xml and the dependency as following: java -jar ivy.jar -settings ivysettings.xml -dependency com.acme mylib "latestRel(17.4)" This works and pulls only the latest artifact with the release 17.4 only but I do not understand why exactly and have the feeling that it works by accident.
        Hide
        jaikiran jaikiran pai added a comment -

        Martin, could you explain your usecase (and/or issue) a bit more, with an example maybe? I read that stackoverflow question but I don't fully follow what the question is.

        Show
        jaikiran jaikiran pai added a comment - Martin , could you explain your usecase (and/or issue) a bit more, with an example maybe? I read that stackoverflow question but I don't fully follow what the question is.
        Hide
        martinm82 Martin added a comment -

        I am having troubles understanding this pattern version matcher. I am looking for a solution that returns me the latest revision based on some specific pattern and couldn't find in the docs a proper explanation how to do that.

        While Google returns me two examples for that use case I am not sure whether the end result is expected or just luck.

        Example:
        https://stackoverflow.com/questions/25549166/latest-stable-revision-with-ivy-excluding-alpha-beta-releases/26213500?noredirect=1#comment81908072_26213500

        If I try this out it seems to work and return me the latest revision but it would be great to get a confirmation that this is expected behaviour.

        Show
        martinm82 Martin added a comment - I am having troubles understanding this pattern version matcher. I am looking for a solution that returns me the latest revision based on some specific pattern and couldn't find in the docs a proper explanation how to do that. While Google returns me two examples for that use case I am not sure whether the end result is expected or just luck. Example: https://stackoverflow.com/questions/25549166/latest-stable-revision-with-ivy-excluding-alpha-beta-releases/26213500?noredirect=1#comment81908072_26213500 If I try this out it seems to work and return me the latest revision but it would be great to get a confirmation that this is expected behaviour.
        Hide
        maartenc Maarten Coene added a comment -

        I've applied your patch. Thanks a lot for the contribution.
        Maarten

        Show
        maartenc Maarten Coene added a comment - I've applied your patch. Thanks a lot for the contribution. Maarten
        Hide
        jkschneider Jon Schneider added a comment -

        Documentation and test case attached. Let me know if there are any problems with the patch formatting.

        Thanks,
        Jon

        Show
        jkschneider Jon Schneider added a comment - Documentation and test case attached. Let me know if there are any problems with the patch formatting. Thanks, Jon

          People

          • Assignee:
            maartenc Maarten Coene
            Reporter:
            xavier Xavier Hanin
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development