Ivy
  1. Ivy
  2. IVY-1248

Module inheritance sometimes fails to locate parent descriptor in deliver process

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.3.0-RC1
    • Component/s: Ant
    • Labels:
      None

      Description

      I recently discovered a bug in Module Inheritance feature (i.e. extends).

       
      <extends organisation="net.foo" module="bar" revision="latest.integration" location="../myParent" extendTypes="configurations,dependencies"/>
      

      The feature works fine if the parent descriptor is already published in your repository, it will use the MRID to retrieve the parent module.
      We also can use a "location" attribute (for dev mode) which defines where to find the parent descriptor. This can make particular sense if you need to share metadatas (dependencies, configurations, whatever) across many subprojects.

      If we get into technical details when "extend" element is found ivy will :

      • check on filesystem if parent module exists (based on location attribute). If no location attribute is found ivy will look in the default one "../ivy.xml".
      • if not found, it will check in the cache using MRID. However it just ignore the version attribute. To be exact it ask for WorkingRevision.
      • if not found it will query repositories using the real MRID (here the version attribute is taken in consideration)
      • if found it will try to flush the parent descriptor in the cache

      Everything works

      But when you invoke ivy:deliver (directly or through ivy:publish) ivy parse the resolved module descriptor in the cache. And the complication comes here.
      Imagine you never publish the "parent module" in the repository. When you will invoke ivy:deliver / ivy:publish ivy will :

      • parse your module descriptor (the one <extend>ing your parent) from the cache
      • while parsing ivy will find the <extend> element and will try to locate the parent descriptor
      • check the location attribute <-- fail because cache pattern is fully configurable in ivy and almost never match with the default value "../ivy.xml"
      • check the cache with the given MRID (organisation="net.foo" module="bar" but with working revision ) <-- almost never find it
      • check the repositories with the real mrid (organisation="net.foo" module="bar" revision="latest.revision") <-- fail as we never published our parent descriptor.
      • deliver process will fail !

      We never really encoutered the problem as this feature was developped in easyant context and we were doing a lot of stuff around this feature. In almost all cases parent descriptor was published by an other mechanism of easyant in a "build scoped repository" (kind of filesystem based repository local to the project).

      While refactoring the code in easyant i just realized that somethings was wrong.

      1. ivy1248-r1033912.patch
        15 kB
        Jean-Louis Boudart

        Activity

        Nicolas Lalevée made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s trunk [ 12313426 ]
        Resolution Fixed [ 1 ]
        Hide
        Nicolas Lalevée added a comment -

        Thanks for the patch and the detailed explanation Jean-Louis !

        Show
        Nicolas Lalevée added a comment - Thanks for the patch and the detailed explanation Jean-Louis !
        Nicolas Lalevée made changes -
        Assignee Nicolas Lalevée [ hibou ]
        Jean-Louis Boudart made changes -
        Attachment ivy1248-r1033912.patch [ 12459343 ]
        Hide
        Jean-Louis Boudart added a comment -

        Please find the attached patch it fixes the issue and contains add related test

        I also refactored a bit the code to make it easier to maintain / extend

        Show
        Jean-Louis Boudart added a comment - Please find the attached patch it fixes the issue and contains add related test I also refactored a bit the code to make it easier to maintain / extend
        Jean-Louis Boudart made changes -
        Field Original Value New Value
        Description I recently discovered a bug in Module Inheritance feature (i.e. extends).

        {code:xml}
        <extends organisation="net.foo" module="bar" revision="latest.integration" location="../myParent" extendTypes="configurations,dependencies"/>
        {code}

        The feature works fine if the parent descriptor is already published in your repository, it will use the MRID to retrieve the parent module.
        We also can use a "location" attribute (for dev mode) which defines where to find the parent descriptor. This can make particular sense if you need to share metadatas (dependencies, configurations, whatever) across many subprojects.

        If we get into technical details when "extend" element is found ivy will :
        * check on filesystem if parent module exists (based on location attribute). If no location attribute is found ivy will look in the default one "../ivy.xml".
        * if not found, it will check in the cache using MRID. However it just ignore the version attribute. To be exact it ask for WorkingRevision.
        * if not found it will query repositories using the real MRID (here the version attribute is taken in consideration)

        Everything works :)

        But when you invoke ivy:deliver (directly or through ivy:publish) ivy parse the resolved module descriptor in the cache. And the complication comes here.
        Imagine you never publish the "parent module" in the repository. When you will invoke ivy:deliver / ivy:publish ivy will :
        * parse your module descriptor (the one <extend>ing your parent) from the cache
        * while parsing ivy will find the <extend> element and will try to locate the parent descriptor
        * check the location attribute <-- fail because cache pattern is fully configurable in ivy and almost never match with the default value "../ivy.xml"
        * check the cache with the given MRID (organisation="net.foo" module="bar" but with working revision ) <-- almost never find
        * check the repositories with the real mrid (organisation="net.foo" module="bar" revision="latest.revision") <-- fail as we never published our parent descriptor.
        * deliver process will fail !

        We never really encoutered the problem as this feature was developped in easyant context and we were doing a lot of stuff around this feature. In almost all cases parent descriptor was published by an other mechanism of easyant in a "build scoped repository" (kind of filesystem based repository local to the project).

        While refactoring the code in easyant i just realized that somethings was wrong.
        I recently discovered a bug in Module Inheritance feature (i.e. extends).

        {code:xml}
        <extends organisation="net.foo" module="bar" revision="latest.integration" location="../myParent" extendTypes="configurations,dependencies"/>
        {code}

        The feature works fine if the parent descriptor is already published in your repository, it will use the MRID to retrieve the parent module.
        We also can use a "location" attribute (for dev mode) which defines where to find the parent descriptor. This can make particular sense if you need to share metadatas (dependencies, configurations, whatever) across many subprojects.

        If we get into technical details when "extend" element is found ivy will :
        * check on filesystem if parent module exists (based on location attribute). If no location attribute is found ivy will look in the default one "../ivy.xml".
        * if not found, it will check in the cache using MRID. However it just ignore the version attribute. To be exact it ask for WorkingRevision.
        * if not found it will query repositories using the real MRID (here the version attribute is taken in consideration)
        * if found it will try to flush the parent descriptor in the cache

        Everything works :)

        But when you invoke ivy:deliver (directly or through ivy:publish) ivy parse the resolved module descriptor in the cache. And the complication comes here.
        Imagine you never publish the "parent module" in the repository. When you will invoke ivy:deliver / ivy:publish ivy will :
        * parse your module descriptor (the one <extend>ing your parent) from the cache
        * while parsing ivy will find the <extend> element and will try to locate the parent descriptor
        * check the location attribute <-- fail because cache pattern is fully configurable in ivy and almost never match with the default value "../ivy.xml"
        * check the cache with the given MRID (organisation="net.foo" module="bar" but with working revision ) <-- almost never find it
        * check the repositories with the real mrid (organisation="net.foo" module="bar" revision="latest.revision") <-- fail as we never published our parent descriptor.
        * deliver process will fail !

        We never really encoutered the problem as this feature was developped in easyant context and we were doing a lot of stuff around this feature. In almost all cases parent descriptor was published by an other mechanism of easyant in a "build scoped repository" (kind of filesystem based repository local to the project).

        While refactoring the code in easyant i just realized that somethings was wrong.
        Hide
        Jean-Louis Boudart added a comment -

        It looks like that the problem is on the way we ask the parent descriptor on cache.
        As explained before when "extend" element is found ivy will :

        • check on filesystem if parent module exists (based on location attribute). If no location attribute is found ivy will look in the default one "../ivy.xml".
        • if not found, it will check in the cache using MRID. However it just ignore the version attribute. To be exact it asks for WorkingRevision.
        • if not found it will query repositories using the real MRID (here the version attribute is taken in consideration)
        • if found it will try to flush the parent descriptor in the cache

        I suggest to update the process like :

        • check revision, if no revision is defined use working revision
        • check on filesystem if parent module exists (based on location attribute). If no location attribute is found ivy will look in the default one "../ivy.xml".
        • if not found, check in the cache using real MRID
        • if not found, query repositories using the real MRID
        • if found it will try to flush the parent descriptor in the cache

        So when deliver process will be started, considering parent descriptor is flushed on cache, and we the check cache use "revision" attribute, ivy will be able to resolve the parent descriptor.

        From the test i've made it seems to solve the problem, so we can consider this as a minor bug to fix.
        I'm going to attach a patch soon

        Show
        Jean-Louis Boudart added a comment - It looks like that the problem is on the way we ask the parent descriptor on cache. As explained before when "extend" element is found ivy will : check on filesystem if parent module exists (based on location attribute). If no location attribute is found ivy will look in the default one "../ivy.xml". if not found, it will check in the cache using MRID. However it just ignore the version attribute. To be exact it asks for WorkingRevision. if not found it will query repositories using the real MRID (here the version attribute is taken in consideration) if found it will try to flush the parent descriptor in the cache I suggest to update the process like : check revision, if no revision is defined use working revision check on filesystem if parent module exists (based on location attribute). If no location attribute is found ivy will look in the default one "../ivy.xml". if not found, check in the cache using real MRID if not found, query repositories using the real MRID if found it will try to flush the parent descriptor in the cache So when deliver process will be started, considering parent descriptor is flushed on cache, and we the check cache use "revision" attribute, ivy will be able to resolve the parent descriptor. From the test i've made it seems to solve the problem, so we can consider this as a minor bug to fix. I'm going to attach a patch soon
        Jean-Louis Boudart created issue -

          People

          • Assignee:
            Nicolas Lalevée
            Reporter:
            Jean-Louis Boudart
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development