Avro
  1. Avro
  2. AVRO-663

avro-tools-1.4.0.jar doesn't meet the maven2 layout standard, making it inaccessble to maven users

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.5.0
    • Component/s: build
    • Labels:
      None

      Description

      in /org/apache/avro/avro/1.4.0 you have avro-tools-1.4.0.jar. This should instead be avro-1.4.0-tools.jar (similar to sources/javadoc jar structure) if this is really just an attached artifact. If this is supposed to be something separate and avro-tools is the artifactId, then it should be in /org/apache/avro/avro-tools/1.4.0/avro-tools-1.4.0.jar

      1. AVRO-663.patch
        16 kB
        Scott Carey
      2. AVRO-663.patch
        14 kB
        Scott Carey

        Activity

        Hide
        Brian Fox added a comment -

        note: i can help move these around if needed, but I need to know the intent of these jars, and then your build needs to be updated to account for future releases.

        Show
        Brian Fox added a comment - note: i can help move these around if needed, but I need to know the intent of these jars, and then your build needs to be updated to account for future releases.
        Hide
        Doug Cutting added a comment -

        The tools jar is a standalone command-line executable that bundles avro and its dependencies.

        Also, FYI, AVRO-647 discusses breaking Avro into more artifacts.

        Show
        Doug Cutting added a comment - The tools jar is a standalone command-line executable that bundles avro and its dependencies. Also, FYI, AVRO-647 discusses breaking Avro into more artifacts.
        Hide
        Scott Carey added a comment -

        Essentially, avro-tools.jar is not for maven users. It is in the maven repository for command line and ant/ivy users.

        avro-tools.jar is simply avro.jar with most of its dependencies bundled (a no-no in maven) and a manifest file to make 'java -jar avro-tools.jar' become a command line tool.
        So, it is a completely different packaging of avro, not an attached artifact.

        Maven users that need to integrate features of the command line tool (such as compiling .avsc and .avpr files to java classes) need a maven plugin, like that proposed in https://issues.apache.org/jira/browse/AVRO-159. This would have its own pom.xml and live in a directory underneath /org/apache/avro.

        Show
        Scott Carey added a comment - Essentially, avro-tools.jar is not for maven users. It is in the maven repository for command line and ant/ivy users. avro-tools.jar is simply avro.jar with most of its dependencies bundled (a no-no in maven) and a manifest file to make 'java -jar avro-tools.jar' become a command line tool. So, it is a completely different packaging of avro, not an attached artifact. Maven users that need to integrate features of the command line tool (such as compiling .avsc and .avpr files to java classes) need a maven plugin, like that proposed in https://issues.apache.org/jira/browse/AVRO-159 . This would have its own pom.xml and live in a directory underneath /org/apache/avro.
        Hide
        Scott Carey added a comment -

        After AVRO-647 we now produce several separate artifacts.

        avro-tools.jar is still somewhat non-standard since it bundles all dependencies inside of it. Perhaps we should have avro-tools.jar only have its code, and have another jar that bundles it all together, as an attachment, e.g. avro-tools-1.5.0-withdeps.jar ?

        Show
        Scott Carey added a comment - After AVRO-647 we now produce several separate artifacts. avro-tools.jar is still somewhat non-standard since it bundles all dependencies inside of it. Perhaps we should have avro-tools.jar only have its code, and have another jar that bundles it all together, as an attachment, e.g. avro-tools-1.5.0-withdeps.jar ?
        Hide
        Thiruvalluvan M. G. added a comment -

        +1

        Some projects call it fatjar. In fact there is a maven plugin for fatjar:

        http://code.google.com/p/maven-fatjar-plugin/

        But I've not used it myself.

        Show
        Thiruvalluvan M. G. added a comment - +1 Some projects call it fatjar. In fact there is a maven plugin for fatjar: http://code.google.com/p/maven-fatjar-plugin/ But I've not used it myself.
        Hide
        Philip Zeyliger added a comment -

        There's also "shade". http://maven.apache.org/plugins/maven-shade-plugin/. I'm not familiar with the distinctions.

        Show
        Philip Zeyliger added a comment - There's also "shade". http://maven.apache.org/plugins/maven-shade-plugin/ . I'm not familiar with the distinctions.
        Hide
        Scott Carey added a comment -

        as far as the shade-plugin there are other things it can do, like move whole dependencies around. For example, we could move paranamer into 'org.apache.avro.paranamer' and prevent jar conflicts. It can also package things into a single jar without moving them around.

        I haven't seen 'fatjar' before. Its worth a look.

        Show
        Scott Carey added a comment - as far as the shade-plugin there are other things it can do, like move whole dependencies around. For example, we could move paranamer into 'org.apache.avro.paranamer' and prevent jar conflicts. It can also package things into a single jar without moving them around. I haven't seen 'fatjar' before. Its worth a look.
        Hide
        Scott Carey added a comment -

        I was wrong, the current, post AVRO-647 avro-tools.jar in trunk does not contain its dependencies. Therefore, it is a standard maven jar without embedded dependencies. We can use 'fatjar', 'shade', or 'assembly' plugins to create another jar with dependencies. This should be done before 1.5.0, I'll address that in this ticket.

        Show
        Scott Carey added a comment - I was wrong, the current, post AVRO-647 avro-tools.jar in trunk does not contain its dependencies. Therefore, it is a standard maven jar without embedded dependencies. We can use 'fatjar', 'shade', or 'assembly' plugins to create another jar with dependencies. This should be done before 1.5.0, I'll address that in this ticket.
        Hide
        Brian Fox added a comment -

        The content of the jar doesn't matter, the name however does if it's in a maven repo. The way it's named now, it's inaccessible via maven or maven aware tools, so why bother putting it into the maven repo?

        Show
        Brian Fox added a comment - The content of the jar doesn't matter, the name however does if it's in a maven repo. The way it's named now, it's inaccessible via maven or maven aware tools, so why bother putting it into the maven repo?
        Hide
        Scott Carey added a comment -

        The new one, for 1.5.0, is maven compliant since the artifact id is 'avro-tools'.

        Show
        Scott Carey added a comment - The new one, for 1.5.0, is maven compliant since the artifact id is 'avro-tools'.
        Hide
        Doug Cutting added a comment -

        The new avro-tools-1.4.0.jar may be compliant, but I also think it's relatively useless. Folks who want to use the command-line tools need an executable jar that includes all of the dependencies. I think we should replace this with a correctly-named avro-1.4.0-tools.jar that includes dependencies. I don't think we need to worry about shading dependencies and such in this case.

        Show
        Doug Cutting added a comment - The new avro-tools-1.4.0.jar may be compliant, but I also think it's relatively useless. Folks who want to use the command-line tools need an executable jar that includes all of the dependencies. I think we should replace this with a correctly-named avro-1.4.0-tools.jar that includes dependencies. I don't think we need to worry about shading dependencies and such in this case.
        Hide
        Scott Carey added a comment -

        This patch modifies the trunk java build to produce an avro-tools-<VERSION>.jar that includes dependencies.

        There are a couple other things I'm working on and this includes a bit of that (part of AVRO-715). I'll have time to separate that work and submit that separately next week.

        Show
        Scott Carey added a comment - This patch modifies the trunk java build to produce an avro-tools-<VERSION>.jar that includes dependencies. There are a couple other things I'm working on and this includes a bit of that (part of AVRO-715 ). I'll have time to separate that work and submit that separately next week.
        Hide
        Doug Cutting added a comment -

        +1 this patch works for me. Thanks!

        Show
        Doug Cutting added a comment - +1 this patch works for me. Thanks!
        Hide
        Scott Carey added a comment -

        Updated patch – minor changes. Removed formatting changes in xml comments and some cruft.

        Show
        Scott Carey added a comment - Updated patch – minor changes. Removed formatting changes in xml comments and some cruft.
        Hide
        Scott Carey added a comment -

        I committed this.

        Show
        Scott Carey added a comment - I committed this.

          People

          • Assignee:
            Scott Carey
            Reporter:
            Brian Fox
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development