Solr
  1. Solr
  2. SOLR-586

Maven - Solr Artifact Publishing

    Details

      Description

      I know there is an issue open (SOLR-19) for getting a solr build going under Maven. This issue differs from that in that it does not concern the build process of the solr project, but rather simple dependency management for maven projects that depend on the solr artifacts. I've outlined a way to easily incorporate solrj + dependencies into your own maven projects, in hopes that others doing this find it useful.

      This issue's purpose is twofold:
      1) Let others know the process.
      2) Open the idea of whether this can be streamlined/incorporated into the standard build in some manner.

      Depending on Solrj in a Maven Project
      1) Build a 1.3 snapshot.
      1.1) Check out the code from http://svn.apache.org/repos/asf/lucene/solr/
      1.2) Build using "ant dist"
      2) Install the artifacts into your maven repo, using the included pom files.
      2.1) Move to your dist/apache-solr-1.3-dev/dist directory.
      2.2) Copy the attached pom files into this directory.
      2.3) Install solr-common into your repo.
      ex) mvn install:install-file -Dfile=apache-solr-common-1.3-dev.jar -DpomFile=solr-common.pom.xml
      2.4) Install solrj into your repo.
      ex) mvn install:install-file -Dfile=apache-solr-solrj-1.3-dev.jar -DpomFile=solrj.pom.xml
      3) Use Solrj in your existing Maven projects by including it as a dependency in your own pom.xml
      <dependency>
      <groupId>org.apache.lucene.solr</groupId>
      <artifactId>solrj</artifactId>
      <version>1.3-SNAPSHOT</version>
      </dependency>

      So given the above process, it seems like it would be relatively simple to standardize this process by:
      1) Including the solr-common and solrj pom files w/ the dist.
      2) Automating the periodic installation of the artifacts to a central repo, such as the ibiblio repo.

      If those steps were performed, then creating a (maven) project based on solrj would be super simple: just #3 from above. Since most custom developments are probably for the clients, it seems like simplifying this would be a nice step to take.

      1. solrj.pom.xml
        3 kB
        Spencer Crissman
      2. solr-common.pom.xml
        2 kB
        Spencer Crissman
      3. solr-dih.pom.xml
        2 kB
        Craig McClanahan
      4. solr-server.pom.xml
        3 kB
        Craig McClanahan
      5. solr2mvn.sh
        1 kB
        Craig McClanahan
      6. SOLR-586-20080811-craigmcc.zip
        13 kB
        Craig McClanahan
      7. SOLR-586.patch
        41 kB
        Shalin Shekhar Mangar
      8. SOLR-586.patch
        44 kB
        Shalin Shekhar Mangar
      9. SOLR-586.patch
        44 kB
        Shalin Shekhar Mangar
      10. SOLR-586-nits.patch
        0.8 kB
        Ryan McKinley

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          So given the above process, it seems like it would be relatively simple to standardize this process by:
          1) Including the solr-common and solrj pom files w/ the dist.
          2) Automating the periodic installation of the artifacts to a central repo, such as the ibiblio repo.

          I don't use Maven, but i'm +1 to the idea.

          FWIW: this is the same basic approach lucene-java took ... most committers were hesitant to rip out the ant build files they were use to in order to use maven to build with, but changes were made to at least generate poms and help automate publishing jars and POMs to the Apache maven repository (which is mirrored by ibiblio)

          Show
          Hoss Man added a comment - So given the above process, it seems like it would be relatively simple to standardize this process by: 1) Including the solr-common and solrj pom files w/ the dist. 2) Automating the periodic installation of the artifacts to a central repo, such as the ibiblio repo. I don't use Maven, but i'm +1 to the idea. FWIW: this is the same basic approach lucene-java took ... most committers were hesitant to rip out the ant build files they were use to in order to use maven to build with, but changes were made to at least generate poms and help automate publishing jars and POMs to the Apache maven repository (which is mirrored by ibiblio)
          Hide
          Craig McClanahan added a comment -

          A third POM to install the Data Import Handler jar file as a separate artifact. This will become obsolete if/when DIH is included in the Solr Common jar.

          Show
          Craig McClanahan added a comment - A third POM to install the Data Import Handler jar file as a separate artifact. This will become obsolete if/when DIH is included in the Solr Common jar.
          Hide
          Craig McClanahan added a comment -

          A shell script to install all three (including the new DIH jar file) Solr jar files into a Maven 2 repository. See comments at the beginning of the script for setup and execution instructions.

          Show
          Craig McClanahan added a comment - A shell script to install all three (including the new DIH jar file) Solr jar files into a Maven 2 repository. See comments at the beginning of the script for setup and execution instructions.
          Hide
          Craig McClanahan added a comment -

          Thanks for the work on this! As I had a need for the same thing (use SolrJ in a Maven based project), it was a happy moment to see someone else had done the basic work for me . To pay back favor for favor, I've added a third POM for the DataImportHandler jar (only needed until DIH is included in the common jar), and a shell script (solr2mvn.sh) that will install all three artifacts for you. See the comments at the beginning of the script for setup and execution instructions.

          Show
          Craig McClanahan added a comment - Thanks for the work on this! As I had a need for the same thing (use SolrJ in a Maven based project), it was a happy moment to see someone else had done the basic work for me . To pay back favor for favor, I've added a third POM for the DataImportHandler jar (only needed until DIH is included in the common jar), and a shell script (solr2mvn.sh) that will install all three artifacts for you. See the comments at the beginning of the script for setup and execution instructions.
          Hide
          Dominic Mitchell added a comment -

          This looks fantastic — it would be really useful for us, we're doing it by hand at present.

          I do have one request though: how hard would it be to attach a source jar as well? The addition to the POM is fairly simple. Most of our projects have this:

            <build>
              <plugins>
                <plugin>
                  <groupId>org.apache.maven.plugins</groupId>
                  <artifactId>maven-source-plugin</artifactId>
                  <executions>
                    <execution>
                      <id>attach-sources</id>
                      <goals>
                        <goal>jar</goal>
                      </goals>
                    </execution>
                  </executions>
                </plugin>
              </plugins>
            </build>
          

          I find that this makes maven artifacts a load more useful.

          Ah, I see the POM isn't actually being used in and of itself. It's just there to list the dependencies. I'd need to amend build.xml to produce source jars as well. Oh well. I'll try and see how much effort that will be.

          Show
          Dominic Mitchell added a comment - This looks fantastic — it would be really useful for us, we're doing it by hand at present. I do have one request though: how hard would it be to attach a source jar as well? The addition to the POM is fairly simple. Most of our projects have this: <build> <plugins> <plugin> <groupId> org.apache.maven.plugins </groupId> <artifactId> maven-source-plugin </artifactId> <executions> <execution> <id> attach-sources </id> <goals> <goal> jar </goal> </goals> </execution> </executions> </plugin> </plugins> </build> I find that this makes maven artifacts a load more useful. Ah, I see the POM isn't actually being used in and of itself. It's just there to list the dependencies. I'd need to amend build.xml to produce source jars as well. Oh well. I'll try and see how much effort that will be.
          Hide
          Craig McClanahan added a comment -

          I do have one request though: how hard would it be to attach a source jar as well?

          I agree that sources jars would be helpful ... and javadoc jars as well, for that matter. As you noted, the poms are not currently being used to build Solr – they are just used for the packaging. I'll take a crack at updating the install script, once the Solr build process produces sources and javadoc jars in addition to the current binary jars.

          Show
          Craig McClanahan added a comment - I do have one request though: how hard would it be to attach a source jar as well? I agree that sources jars would be helpful ... and javadoc jars as well, for that matter. As you noted, the poms are not currently being used to build Solr – they are just used for the packaging. I'll take a crack at updating the install script, once the Solr build process produces sources and javadoc jars in addition to the current binary jars.
          Hide
          Craig McClanahan added a comment -

          Additional POM for the server part of Apache Solr.

          Show
          Craig McClanahan added a comment - Additional POM for the server part of Apache Solr.
          Hide
          Craig McClanahan added a comment -

          Updated script that copies the server POM and JAR files as well.

          Show
          Craig McClanahan added a comment - Updated script that copies the server POM and JAR files as well.
          Hide
          Shalin Shekhar Mangar added a comment -

          A third POM to install the Data Import Handler jar file as a separate artifact. This will become obsolete if/when DIH is included in the Solr Common jar.

          DIH is a contrib project which means it will always be a separate artifact.

          I agree that sources jars would be helpful ... and javadoc jars as well, for that matter

          I agree that source and javadoc jars will be useful. Javadocs are already generated separately for solrj and DIH but not as jars. The solr-commons and solr core javadocs are combined together currently. We will need modifications to build files to generate javadoc and source jars for each of the artifacts separately.

          I'll take a crack at updating the install script, once the Solr build process produces sources and javadoc jars in addition to the current binary jars.

          I think it would be better to integrate maven deploy with the ant builds similar to how Lucene does it.

          I'm going to start making the changes. Hope to post a patch soon.

          Show
          Shalin Shekhar Mangar added a comment - A third POM to install the Data Import Handler jar file as a separate artifact. This will become obsolete if/when DIH is included in the Solr Common jar. DIH is a contrib project which means it will always be a separate artifact. I agree that sources jars would be helpful ... and javadoc jars as well, for that matter I agree that source and javadoc jars will be useful. Javadocs are already generated separately for solrj and DIH but not as jars. The solr-commons and solr core javadocs are combined together currently. We will need modifications to build files to generate javadoc and source jars for each of the artifacts separately. I'll take a crack at updating the install script, once the Solr build process produces sources and javadoc jars in addition to the current binary jars. I think it would be better to integrate maven deploy with the ant builds similar to how Lucene does it. I'm going to start making the changes. Hope to post a patch soon.
          Hide
          Craig McClanahan added a comment -

          I'm going to start making the changes. Hope to post a patch soon.

          Cool. Two things I noticed that we should pay attention to:

          • The solr-server POM I posted should also declare dependencies on the Lucene
            2.4-DEV jars that it uses (which also implies you need to have installed those
            locally by running "ant generate-maven-artifacts" in the Lucene source tree).
          • Given we need to do that, it would be nice to add a similar build target into the
            Solr build.xml file, so the prototypes get things like the right version numbers.
            This will also require the same prerequesite that the Lucene build does, that
            you have installed the Ant Maven Tasks jar into your Ant environment.

          Does SolrJ also require a Lucene dependency if you use it in the "embedded server" style?

          Show
          Craig McClanahan added a comment - I'm going to start making the changes. Hope to post a patch soon. Cool. Two things I noticed that we should pay attention to: The solr-server POM I posted should also declare dependencies on the Lucene 2.4-DEV jars that it uses (which also implies you need to have installed those locally by running "ant generate-maven-artifacts" in the Lucene source tree). Given we need to do that, it would be nice to add a similar build target into the Solr build.xml file, so the prototypes get things like the right version numbers. This will also require the same prerequesite that the Lucene build does, that you have installed the Ant Maven Tasks jar into your Ant environment. Does SolrJ also require a Lucene dependency if you use it in the "embedded server" style?
          Hide
          Shalin Shekhar Mangar added a comment -

          The solr-server POM I posted should also declare dependencies on the Lucene 2.4-DEV jars that it uses (which also implies you need to have installed those locally by running "ant generate-maven-artifacts" in the Lucene source tree).

          I'm not sure how we will manage the dependency of lucene jars. It would be wrong to declare a dependency on lucene snapshot jars since Solr has a particular revision of the trunk checked in. We should probably hold on to the solr-server for now and focus on SolrJ (without embedded) and DIH working. The DIH artifact should only depend on solr-commons because this jar will only be used for extending DIH API.

          Show
          Shalin Shekhar Mangar added a comment - The solr-server POM I posted should also declare dependencies on the Lucene 2.4-DEV jars that it uses (which also implies you need to have installed those locally by running "ant generate-maven-artifacts" in the Lucene source tree). I'm not sure how we will manage the dependency of lucene jars. It would be wrong to declare a dependency on lucene snapshot jars since Solr has a particular revision of the trunk checked in. We should probably hold on to the solr-server for now and focus on SolrJ (without embedded) and DIH working. The DIH artifact should only depend on solr-commons because this jar will only be used for extending DIH API.
          Hide
          Craig McClanahan added a comment -

          I'm not sure how we will manage the dependency of lucene jars. It would be wrong to declare a dependency on lucene snapshot jars since Solr has a particular revision of the trunk checked in.

          The typical Maven approach to this would be to make each of the specific Lucene jars Solr depends on (the ones checked in to the "lib" directory) into Solr artifacts (groupId of org.apache.lucene.solr), with version number that correspond to the Solr version they are a part of. That way, there's absolutely no interference with Lucene's own Maven artifacts, and no dependence on Lucene SNAPSHOT versions which can change underneath us.

          Show
          Craig McClanahan added a comment - I'm not sure how we will manage the dependency of lucene jars. It would be wrong to declare a dependency on lucene snapshot jars since Solr has a particular revision of the trunk checked in. The typical Maven approach to this would be to make each of the specific Lucene jars Solr depends on (the ones checked in to the "lib" directory) into Solr artifacts (groupId of org.apache.lucene.solr), with version number that correspond to the Solr version they are a part of. That way, there's absolutely no interference with Lucene's own Maven artifacts, and no dependence on Lucene SNAPSHOT versions which can change underneath us.
          Hide
          Craig McClanahan added a comment -

          Heres a substantially updated version of the earlier attachments, based in large part on the approach the Lucene folks took (creds to them for doing the grunt work . To use it:

          • Download and unzip the patch into the top level directory of
            your Solr source tree from SVN. This will create a "maven"
            subdirectory with the new files.
          • Execute "ant clean dist" to create the usual Solr artifacts.
          • Change directory to "maven", then execute "ant -f build-maven.xml"
            to create the Maven artifacts and install them on your local repository.
            This includes the Lucene variants checked in to the "lib" directory,
            but done in a way that won't interfere with Lucene's own artifacts
            when 2.4 goes final.

          Shalin, you're welcome to use whatever of this makes sense – but this functionality scratched my own personal itch quite nicely (Maven based app that needed Solrj in embedded mode).

          Show
          Craig McClanahan added a comment - Heres a substantially updated version of the earlier attachments, based in large part on the approach the Lucene folks took (creds to them for doing the grunt work . To use it: Install the Maven Ant Tasks in a place accessible to your Ant (see < http://maven.apache.org/ant-tasks.html > for more info). Download and unzip the patch into the top level directory of your Solr source tree from SVN. This will create a "maven" subdirectory with the new files. Execute "ant clean dist" to create the usual Solr artifacts. Change directory to "maven", then execute "ant -f build-maven.xml" to create the Maven artifacts and install them on your local repository. This includes the Lucene variants checked in to the "lib" directory, but done in a way that won't interfere with Lucene's own artifacts when 2.4 goes final. Shalin, you're welcome to use whatever of this makes sense – but this functionality scratched my own personal itch quite nicely (Maven based app that needed Solrj in embedded mode).
          Hide
          Craig McClanahan added a comment -

          Hmm ... JIRA doesn't make it at all obvious that the previous comment refers to the attachment named <SOLR-586-20080811-craigmcc.zip> even though they were uploaded together ... so this should clarify things.

          Show
          Craig McClanahan added a comment - Hmm ... JIRA doesn't make it at all obvious that the previous comment refers to the attachment named < SOLR-586 -20080811-craigmcc.zip> even though they were uploaded together ... so this should clarify things.
          Hide
          Shalin Shekhar Mangar added a comment -

          This is awesome! Thanks!

          I'm integrating it with the existing builds so that there would be no need for a maven folder.

          Show
          Shalin Shekhar Mangar added a comment - This is awesome! Thanks! I'm integrating it with the existing builds so that there would be no need for a maven folder.
          Hide
          Shalin Shekhar Mangar added a comment -

          This patch (SOLR-586.patch) integrates all of Craig's POM files and build targets into the Solr build.xml and directory structure. Solr uses a particular revision of commons-csv for which I've created a POM just like Craig did for the lucene jars. The group name for all artifacts is changed to org.apache.solr.

          Another thing I noticed was that we are using a dev-snapshot of stax. Any particular reason for this? Should we move to a stable version? 1.2.0 is the latest stable version I can find on the central maven repository.

          Show
          Shalin Shekhar Mangar added a comment - This patch ( SOLR-586 .patch) integrates all of Craig's POM files and build targets into the Solr build.xml and directory structure. Solr uses a particular revision of commons-csv for which I've created a POM just like Craig did for the lucene jars. The group name for all artifacts is changed to org.apache.solr. Another thing I noticed was that we are using a dev-snapshot of stax. Any particular reason for this? Should we move to a stable version? 1.2.0 is the latest stable version I can find on the central maven repository.
          Hide
          Ryan McKinley added a comment -

          Another thing I noticed was that we are using a dev-snapshot of stax. Any particular reason for this? Should we move to a stable version? 1.2.0 is the latest stable version I can find on the central maven repository.

          yes, if possible, we should try to avoid dev builds. However, we should switch the .jar in an independent issue

          Show
          Ryan McKinley added a comment - Another thing I noticed was that we are using a dev-snapshot of stax. Any particular reason for this? Should we move to a stable version? 1.2.0 is the latest stable version I can find on the central maven repository. yes, if possible, we should try to avoid dev builds. However, we should switch the .jar in an independent issue
          Hide
          Shalin Shekhar Mangar added a comment - - edited

          Changes

          • Reverted back to org.apache.lucene.solr as the group name due to Maven central repository's restriction on having groupId same as the controlled domain name.
          • Added target dist-src which packages source code for each artifact
          • Added target dist-javadoc which packages javadoc for each artifact
          • Renamed the main solr jar to apache-solr-server-$ {version}

            .jar for consistency with other jars

          • generate-maven-artifact target also adds source and javadocs for each artifact

          If we want to publish nightly maven snapshots, then after executing the main nightly target, we must execute the following on hudson:
          {{

          {ant clean generate-maven-artifacts}

          }}

          It will be OK to put a dependency on generate-maven-artifacts in the nightly target. Ofcourse, first we need to setup a build.property file on hudson which gives the URL to maven repo, username and private key. Someone who is familar with how Lucene did this and has karma on hudson will need to take this up now.

          When the artifacts are published, the following should be added to your pom.xml to use solrj

          <dependency>
            <groupId>org.apache.lucene.solr</groupId>
            <artifactId>solr-solrj</artifactId>
            <version>1.3-SNAPSHOT</version>
          </dependency>
          

          To use the EmbeddedSolrServer, a dependency to solr-server must also be added:

          <dependency>
            <groupId>org.apache.lucene.solr</groupId>
            <artifactId>solr-core</artifactId>
            <version>1.3-SNAPSHOT</version>
          </dependency>
          

          I've not added a dependency on solr-server for SolrJ because it is only needed for EmbeddedSolrServer but not if you only use CommonsHttpSolrServer.

          This is ready to be committed now.

          Show
          Shalin Shekhar Mangar added a comment - - edited Changes Reverted back to org.apache.lucene.solr as the group name due to Maven central repository's restriction on having groupId same as the controlled domain name. Added target dist-src which packages source code for each artifact Added target dist-javadoc which packages javadoc for each artifact Renamed the main solr jar to apache-solr-server-$ {version} .jar for consistency with other jars generate-maven-artifact target also adds source and javadocs for each artifact If we want to publish nightly maven snapshots, then after executing the main nightly target, we must execute the following on hudson: {{ {ant clean generate-maven-artifacts} }} It will be OK to put a dependency on generate-maven-artifacts in the nightly target. Ofcourse, first we need to setup a build.property file on hudson which gives the URL to maven repo, username and private key. Someone who is familar with how Lucene did this and has karma on hudson will need to take this up now. When the artifacts are published, the following should be added to your pom.xml to use solrj <dependency> <groupId> org.apache.lucene.solr </groupId> <artifactId> solr-solrj </artifactId> <version> 1.3-SNAPSHOT </version> </dependency> To use the EmbeddedSolrServer, a dependency to solr-server must also be added: <dependency> <groupId> org.apache.lucene.solr </groupId> <artifactId> solr-core </artifactId> <version> 1.3-SNAPSHOT </version> </dependency> I've not added a dependency on solr-server for SolrJ because it is only needed for EmbeddedSolrServer but not if you only use CommonsHttpSolrServer. This is ready to be committed now.
          Hide
          Craig McClanahan added a comment -

          The patch fails to apply cleanly for me ... the changes to "contrib/dataimporthandler/build.xml" fail at lines 51 and 61. Did something change in this file in parallel to your creating the patch? (I'm at r685285).

          I've not added a dependency on solr-server for SolrJ because it is only needed for EmbeddedSolrServer but not if you only use CommonsHttpSolrServer.

          +1.

          Show
          Craig McClanahan added a comment - The patch fails to apply cleanly for me ... the changes to "contrib/dataimporthandler/build.xml" fail at lines 51 and 61. Did something change in this file in parallel to your creating the patch? (I'm at r685285). I've not added a dependency on solr-server for SolrJ because it is only needed for EmbeddedSolrServer but not if you only use CommonsHttpSolrServer. +1.
          Hide
          Shalin Shekhar Mangar added a comment -

          Can you update to the latest trunk and try again?

          It applies cleanly on the trunk.

          Show
          Shalin Shekhar Mangar added a comment - Can you update to the latest trunk and try again? It applies cleanly on the trunk.
          Hide
          Craig McClanahan added a comment -

          Can you update to the latest trunk and try again?

          Bad News: Even with a fresh checkout of the source, the patch fails to apply.

          Good News: This looks like a botch in the patch file itself; there's a missing "-" to delete the previous line 53 of the old "contrib/dataimporthandler/build.xml". OK, let's fix it manually and try the build.

          Bad News: "ant clean generate-maven-artifacts" fails. This time it's because the patched version of the top-level build.xml is expecting the POM templates to be in either "lib" or the top level directories for contrib stuff, where they are actually all in the top level directory. Let's fix that as well.

          Best News: Everything works.

          I suspect that the second patch file doesn't necessarily reflect all the most recent changes. My recommendation is to go ahead and commit the patch as it actually exists in Shalin's tree, and then clean up anything that might still be missed up. We shouldn't disrupt anything if the nightly build script hasn't been changed yet. I can commit to check any newly committed change first thing tomorrow (Pacific time).

          Show
          Craig McClanahan added a comment - Can you update to the latest trunk and try again? Bad News: Even with a fresh checkout of the source, the patch fails to apply. Good News: This looks like a botch in the patch file itself; there's a missing "-" to delete the previous line 53 of the old "contrib/dataimporthandler/build.xml". OK, let's fix it manually and try the build. Bad News: "ant clean generate-maven-artifacts" fails. This time it's because the patched version of the top-level build.xml is expecting the POM templates to be in either "lib" or the top level directories for contrib stuff, where they are actually all in the top level directory. Let's fix that as well. Best News: Everything works. I suspect that the second patch file doesn't necessarily reflect all the most recent changes. My recommendation is to go ahead and commit the patch as it actually exists in Shalin's tree, and then clean up anything that might still be missed up. We shouldn't disrupt anything if the nightly build script hasn't been changed yet. I can commit to check any newly committed change first thing tomorrow (Pacific time).
          Hide
          Shalin Shekhar Mangar added a comment -

          Are you sure you are using the latest SOLR-586.patch ? I just tried it on a clean checkout and it works well.

          Show
          Shalin Shekhar Mangar added a comment - Are you sure you are using the latest SOLR-586 .patch ? I just tried it on a clean checkout and it works well.
          Hide
          Shalin Shekhar Mangar added a comment -

          Ok, my bad. It doesn't work. I should stop using IDEs to create/apply patches. I'll post a new one shortly.

          Show
          Shalin Shekhar Mangar added a comment - Ok, my bad. It doesn't work. I should stop using IDEs to create/apply patches. I'll post a new one shortly.
          Hide
          Shalin Shekhar Mangar added a comment -

          This one should be correct. Sorry for all the confusion I caused.

          Show
          Shalin Shekhar Mangar added a comment - This one should be correct. Sorry for all the confusion I caused.
          Hide
          Craig McClanahan added a comment -

          I still have the same problems as reported earlier (patch doesn't apply, build.xml changes reflect "lib/xxx" paths for the templates), so it still fails for me. However, I still recommend that we go ahead and apply the actual commits, and clean up any problems later.

          Show
          Craig McClanahan added a comment - I still have the same problems as reported earlier (patch doesn't apply, build.xml changes reflect "lib/xxx" paths for the templates), so it still fails for me. However, I still recommend that we go ahead and apply the actual commits, and clean up any problems later.
          Hide
          Shalin Shekhar Mangar added a comment -

          Try adding -p0. I have the same problem if I don't use -p0 in the patch command.

          patch -p0 < SOLR-586.patch

          Show
          Shalin Shekhar Mangar added a comment - Try adding -p0. I have the same problem if I don't use -p0 in the patch command. patch -p0 < SOLR-586 .patch
          Hide
          Shalin Shekhar Mangar added a comment -

          Committed revision 685497.

          Thanks Spencer and Craig!

          I'll keep this issue open for a day or two in case any problems show up with the builds or until we can make the necessary changes on hudson.

          Show
          Shalin Shekhar Mangar added a comment - Committed revision 685497. Thanks Spencer and Craig! I'll keep this issue open for a day or two in case any problems show up with the builds or until we can make the necessary changes on hudson.
          Hide
          Ryan McKinley added a comment -

          Is this expected? Or do we each need to download and install the maven ant task?

          bicho:s3 ryan$ ant generate-maven-artifacts
          Buildfile: build.xml
          
          maven.ant.tasks-check:
          
          BUILD FAILED
          /Users/ryan/Documents/workspace/APACHE/s3/common-build.xml:334:
          ##################################################################
          Maven ant tasks not found.
          Please make sure the maven-ant-tasks jar is in ANT_HOME/lib, or made
          available to Ant using other mechanisms like -lib or CLASSPATH.
          ##################################################################
          
          Show
          Ryan McKinley added a comment - Is this expected? Or do we each need to download and install the maven ant task? bicho:s3 ryan$ ant generate-maven-artifacts Buildfile: build.xml maven.ant.tasks-check: BUILD FAILED /Users/ryan/Documents/workspace/APACHE/s3/common-build.xml:334: ################################################################## Maven ant tasks not found. Please make sure the maven-ant-tasks jar is in ANT_HOME/lib, or made available to Ant using other mechanisms like -lib or CLASSPATH. ##################################################################
          Hide
          Shalin Shekhar Mangar added a comment -

          Yes, this is expected. You must have the maven-ant-tasks jar in your classpath to use the maven-specific ant targets.

          We must publish jars ourselves (through hudson for snapshots and through a one-time script for the release). Once that is done, these steps will not be needed for any user who wants to use the artifacts in his maven project.

          Show
          Shalin Shekhar Mangar added a comment - Yes, this is expected. You must have the maven-ant-tasks jar in your classpath to use the maven-specific ant targets. We must publish jars ourselves (through hudson for snapshots and through a one-time script for the release). Once that is done, these steps will not be needed for any user who wants to use the artifacts in his maven project.
          Hide
          Shalin Shekhar Mangar added a comment -

          One important point to note is that along with Solr artifacts, we will also publish un-released pieces of code (Lucene 2.4-dev and commons-csv snapshots) as maven artifacts. Though they will be under Solr's groupId, but they will still be available for people to use through the central repository.

          I'm not sure if that is OK with Apache's release policies.

          Show
          Shalin Shekhar Mangar added a comment - One important point to note is that along with Solr artifacts, we will also publish un-released pieces of code (Lucene 2.4-dev and commons-csv snapshots) as maven artifacts. Though they will be under Solr's groupId, but they will still be available for people to use through the central repository. I'm not sure if that is OK with Apache's release policies.
          Hide
          Craig McClanahan added a comment -

          Committed revision 685497.

          This worked for me. Thanks!

          bp. One important point to note is that along with Solr artifacts, we will also publish un-released pieces of code (Lucene 2.4-dev and commons-csv snapshots) as maven artifacts. Though they will be under Solr's groupId, but they will still be available for people to use through the central repository.

          Well, technically releasing Solr 1.3 will release these artifacts (they'll be branded as part of Solr version 1.3, not Lucene 2.4-dev, in the repository). In this particular case, it's even the same PMC, so presumably the Lucene Java folks will be in agreement with this if the vote passes. If that wasn't the case, the normal license compatibility rules would apply.

          Show
          Craig McClanahan added a comment - Committed revision 685497. This worked for me. Thanks! bp. One important point to note is that along with Solr artifacts, we will also publish un-released pieces of code (Lucene 2.4-dev and commons-csv snapshots) as maven artifacts. Though they will be under Solr's groupId, but they will still be available for people to use through the central repository. Well, technically releasing Solr 1.3 will release these artifacts (they'll be branded as part of Solr version 1.3, not Lucene 2.4-dev, in the repository). In this particular case, it's even the same PMC, so presumably the Lucene Java folks will be in agreement with this if the vote passes. If that wasn't the case, the normal license compatibility rules would apply.
          Hide
          Ryan McKinley added a comment -

          minor changes:

          1. change the inceptionYear from 2000 -> 2006 (when solr donated to Apache)
          2. removed "Embedded" from the core description.

          Show
          Ryan McKinley added a comment - minor changes: 1. change the inceptionYear from 2000 -> 2006 (when solr donated to Apache) 2. removed "Embedded" from the core description.
          Hide
          Shalin Shekhar Mangar added a comment -

          Committed revision 685781.

          Thanks Ryan!

          Show
          Shalin Shekhar Mangar added a comment - Committed revision 685781. Thanks Ryan!
          Hide
          Ryan McKinley added a comment -

          I'm running into a funny issue.

          $ ant clean
          $ ant generate-maven-artifacts

          everything works great

          now run: ant generate-maven-artifacts again. The second time, I get:

          BUILD FAILED
          /Users/ryan/Documents/workspace/APACHE/s3/build.xml:645: A tar file cannot include itself

          not really a big deal, but figured I'd point it out...

          Show
          Ryan McKinley added a comment - I'm running into a funny issue. $ ant clean $ ant generate-maven-artifacts everything works great now run: ant generate-maven-artifacts again. The second time, I get: BUILD FAILED /Users/ryan/Documents/workspace/APACHE/s3/build.xml:645: A tar file cannot include itself not really a big deal, but figured I'd point it out...
          Hide
          Shalin Shekhar Mangar added a comment -

          Nothing to do with generate-maven-artifacts target. Run ant package twice and the same thing happens. Simply deleting the tar/zip before package runs will fix this. I shall make the change in the package target.

          Show
          Shalin Shekhar Mangar added a comment - Nothing to do with generate-maven-artifacts target. Run ant package twice and the same thing happens. Simply deleting the tar/zip before package runs will fix this. I shall make the change in the package target.
          Hide
          Ryan McKinley added a comment -

          the included pom is missing a reference to commons-fileupload. This is needed in the core and embedded solrj

          Index: client/java/solrj/solr-solrj-pom.xml.template
          ===================================================================
          --- client/java/solrj/solr-solrj-pom.xml.template	(revision 686022)
          +++ client/java/solrj/solr-solrj-pom.xml.template	(working copy)
          @@ -66,6 +66,11 @@
                 <artifactId>commons-logging</artifactId>
                 <version>1.0.4</version>
               </dependency>
          +    <dependency>
          +      <groupId>commons-fileupload</groupId>
          +      <artifactId>commons-fileupload</artifactId>
          +      <version>1.2</version>
          +    </dependency>
           
               <!-- Stax -->
               <dependency>
          
          
          Show
          Ryan McKinley added a comment - the included pom is missing a reference to commons-fileupload. This is needed in the core and embedded solrj Index: client/java/solrj/solr-solrj-pom.xml.template =================================================================== --- client/java/solrj/solr-solrj-pom.xml.template (revision 686022) +++ client/java/solrj/solr-solrj-pom.xml.template (working copy) @@ -66,6 +66,11 @@ <artifactId>commons-logging</artifactId> <version>1.0.4</version> </dependency> + <dependency> + <groupId>commons-fileupload</groupId> + <artifactId>commons-fileupload</artifactId> + <version>1.2</version> + </dependency> <!-- Stax --> <dependency>
          Hide
          Shalin Shekhar Mangar added a comment -

          Committed revision 686039.

          Sorry, my bad. Thanks Ryan!

          Show
          Shalin Shekhar Mangar added a comment - Committed revision 686039. Sorry, my bad. Thanks Ryan!
          Hide
          Grant Ingersoll added a comment -

          Just an FYI, I had the Maven 2.0.7 Ant tasks and got:

          generate-maven-artifacts:
          [mkdir] Created dir: ../solr-clean/build/maven
          [mkdir] Created dir: .../solr-clean/dist/maven
          [copy] Copying 1 file to .../solr-clean/build/maven/src/maven
          [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2:runtime

          BUILD FAILED
          .../solr-clean/build.xml:695: The following error occurred while executing this line:
          ....solr-clean/common-build.xml:252: artifact:deploy doesn't support the "uniqueVersion" attribute

          Upgrading them to 2.0.9 and it works.

          Show
          Grant Ingersoll added a comment - Just an FYI, I had the Maven 2.0.7 Ant tasks and got: generate-maven-artifacts: [mkdir] Created dir: ../solr-clean/build/maven [mkdir] Created dir: .../solr-clean/dist/maven [copy] Copying 1 file to .../solr-clean/build/maven/src/maven [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2:runtime BUILD FAILED .../solr-clean/build.xml:695: The following error occurred while executing this line: ....solr-clean/common-build.xml:252: artifact:deploy doesn't support the "uniqueVersion" attribute Upgrading them to 2.0.9 and it works.
          Hide
          Grant Ingersoll added a comment -

          Can we close this?

          Show
          Grant Ingersoll added a comment - Can we close this?
          Hide
          Shalin Shekhar Mangar added a comment -

          The uniqueVersion="false" forces maven to use the same version each time overwriting the artifacts. No need to create a separate snapshot artifact each time – it will just clog the repository.

          Can we close this?

          I've kept it open just to make sure we remember to start publishing snapshot artifacts before 1.3 and of course released artifact when 1.3 goes out. Some one with enough karma should take a look at lucene's way of doing this and make the changes on hudson.

          Show
          Shalin Shekhar Mangar added a comment - The uniqueVersion="false" forces maven to use the same version each time overwriting the artifacts. No need to create a separate snapshot artifact each time – it will just clog the repository. Can we close this? I've kept it open just to make sure we remember to start publishing snapshot artifacts before 1.3 and of course released artifact when 1.3 goes out. Some one with enough karma should take a look at lucene's way of doing this and make the changes on hudson.
          Hide
          Grant Ingersoll added a comment -

          I'll see through the Hudson stuff to take care of the publishing aspects.

          Show
          Grant Ingersoll added a comment - I'll see through the Hudson stuff to take care of the publishing aspects.
          Hide
          Grant Ingersoll added a comment -

          I changed the group id to be org.apache.solr per Maven's recommendation at http://maven.apache.org/pom.html#Maven_Coordinates

          Committed revision 687158.

          Show
          Grant Ingersoll added a comment - I changed the group id to be org.apache.solr per Maven's recommendation at http://maven.apache.org/pom.html#Maven_Coordinates Committed revision 687158.
          Hide
          Grant Ingersoll added a comment -

          Have a look at: http://people.apache.org/maven-snapshot-repository/org/apache/solr/

          I'll update the wiki w/ details, too, but here's the gist:

          Hudson builds the artifacts via the generate-maven-artifacts. A cron job runs on my account on Hudson (just like for Lucene) and copies the artifacts to the snapshot repo on p.a.o.

          Then, the only thing that remains is publishing the artifacts to the official Maven repo on the release. I will update the release instructions to do this, per the Lucene Java instructions.

          Show
          Grant Ingersoll added a comment - Have a look at: http://people.apache.org/maven-snapshot-repository/org/apache/solr/ I'll update the wiki w/ details, too, but here's the gist: Hudson builds the artifacts via the generate-maven-artifacts. A cron job runs on my account on Hudson (just like for Lucene) and copies the artifacts to the snapshot repo on p.a.o. Then, the only thing that remains is publishing the artifacts to the official Maven repo on the release. I will update the release instructions to do this, per the Lucene Java instructions.
          Hide
          Shalin Shekhar Mangar added a comment -

          Wow, this is great! Thanks Grant!

          Show
          Shalin Shekhar Mangar added a comment - Wow, this is great! Thanks Grant!
          Hide
          Grant Ingersoll added a comment -

          OK, I think we are in pretty good shape. I updated the Wiki under the committers section, published an initial nightly build.

          Spencer, et. al. please check it out and let me know if we can close this. Obviously, we can't publish the officially released artifacts until after 1.3, so if there is an error there, we will have to reopen the issue.

          Show
          Grant Ingersoll added a comment - OK, I think we are in pretty good shape. I updated the Wiki under the committers section, published an initial nightly build. Spencer, et. al. please check it out and let me know if we can close this. Obviously, we can't publish the officially released artifacts until after 1.3, so if there is an error there, we will have to reopen the issue.
          Hide
          Grant Ingersoll added a comment -

          Maven artifacts are now being published per the link above. Official release artifacts will be published at that time.

          Show
          Grant Ingersoll added a comment - Maven artifacts are now being published per the link above. Official release artifacts will be published at that time.
          Hide
          David Smiley added a comment -

          This hasn't worked for quite some time. Should we re-open this or open a new issue?

          Show
          David Smiley added a comment - This hasn't worked for quite some time. Should we re-open this or open a new issue?

            People

            • Assignee:
              Grant Ingersoll
              Reporter:
              Spencer Crissman
            • Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development