Details

    • Type: Task Task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.7.1, 0.8.0
    • Component/s: Build Infrastructure
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Getting hive into maven main repositories

      Documentation on how to do this is on:
      http://maven.apache.org/guides/mini/guide-central-repository-upload.html

      1. make-maven.log
        42 kB
        Ning Zhang
      2. hiveReleasedToMaven.tar.gz
        6 kB
        Gerrit Jansen van Vuuren
      3. HIVE-1095-trunk.patch
        48 kB
        Gerrit Jansen van Vuuren
      4. HIVE-1095.v6.patch
        35 kB
        Amareshwari Sriramadasu
      5. HIVE-1095.v5.PATCH
        27 kB
        Gerrit Jansen van Vuuren
      6. HIVE-1095.v4.PATCH
        27 kB
        Gerrit Jansen van Vuuren
      7. HIVE-1095.v3.PATCH
        133 kB
        Gerrit Jansen van Vuuren
      8. HIVE-1095.v2.PATCH
        131 kB
        Gerrit Jansen van Vuuren
      9. HIVE-1095.7.patch.txt
        24 kB
        Carl Steinbach

        Issue Links

          Activity

          Gerrit Jansen van Vuuren created issue -
          Gerrit Jansen van Vuuren made changes -
          Field Original Value New Value
          Fix Version/s 0.4.1 [ 12314225 ]
          Affects Version/s 0.4.1 [ 12314225 ]
          Hide
          Gerrit Jansen van Vuuren added a comment -

          OK, I've given some thought on this and propose the following:

          Have a task called make-pom in the build-common.xml that uses the ivy makepom ant task to auto generate a pom for each project:

          The ant task is:

          <target name="make-pom">
          <ivy:makepom ivyfile="$

          {basedir}/ivy.xml" pomfile="${basedir}

          /pom.xml">
          <mapping conf="default" scope="compile"/>
          <mapping conf="runtime" scope="runtime"/>
          </ivy:makepom>
          </target>

          The generated pom for CLI is:
          <?xml version="1.0" encoding="UTF-8"?>
          <!--
          Apache Maven 2 POM generated by Apache Ivy
          http://ant.apache.org/ivy/
          Apache Ivy version: 2.1.0 20090925235825
          -->
          <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

          <modelVersion>4.0.0</modelVersion>
          <groupId>org.apache.hadoop.hive</groupId>
          <artifactId>cli</artifactId>
          <packaging>jar</packaging>
          <version>0.4.1</version>
          <dependencies>
          <dependency>
          <groupId>hadoop</groupId>
          <artifactId>core</artifactId>
          <version>0.19.0</version>
          <optional>true</optional>
          </dependency>
          </dependencies>
          </project>

          From here I can make an ant task that will load the JAR and POM for all the hive subprojects into a maven2 repo.

          The next step would be to setup a hive maven repo.
          Basically how this works is:
          Hive has its own repo and makes ONLY releases to this.
          Then send a request (I can do that) to the maven repos to be synched up with then.
          And voila, you'll have all hive releases that are released to the hive maven repo available on the main maven repos.

          Does your team have access to a server to do this setup?
          Below I write a mini how to for artifactory:
          Easiest is as standalone:
          -> Download the artifactory zip at http://sourceforge.net/projects/artifactory/
          -> run bin/artifactory.sh
          As a WAR:
          -> Download the zip file
          -> copy to webapps/artifactory.war to tomcat or jetty.

          Setup is done via the artifactory admin window:
          defaults are: username: admin password: password (must be changed )

          Show
          Gerrit Jansen van Vuuren added a comment - OK, I've given some thought on this and propose the following: Have a task called make-pom in the build-common.xml that uses the ivy makepom ant task to auto generate a pom for each project: The ant task is: <target name="make-pom"> <ivy:makepom ivyfile="$ {basedir}/ivy.xml" pomfile="${basedir} /pom.xml"> <mapping conf="default" scope="compile"/> <mapping conf="runtime" scope="runtime"/> </ivy:makepom> </target> The generated pom for CLI is: <?xml version="1.0" encoding="UTF-8"?> <!-- Apache Maven 2 POM generated by Apache Ivy http://ant.apache.org/ivy/ Apache Ivy version: 2.1.0 20090925235825 --> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd "> <modelVersion>4.0.0</modelVersion> <groupId>org.apache.hadoop.hive</groupId> <artifactId>cli</artifactId> <packaging>jar</packaging> <version>0.4.1</version> <dependencies> <dependency> <groupId>hadoop</groupId> <artifactId>core</artifactId> <version>0.19.0</version> <optional>true</optional> </dependency> </dependencies> </project> From here I can make an ant task that will load the JAR and POM for all the hive subprojects into a maven2 repo. The next step would be to setup a hive maven repo. Basically how this works is: Hive has its own repo and makes ONLY releases to this. Then send a request (I can do that) to the maven repos to be synched up with then. And voila, you'll have all hive releases that are released to the hive maven repo available on the main maven repos. Does your team have access to a server to do this setup? Below I write a mini how to for artifactory: Easiest is as standalone: -> Download the artifactory zip at http://sourceforge.net/projects/artifactory/ -> run bin/artifactory.sh As a WAR: -> Download the zip file -> copy to webapps/artifactory.war to tomcat or jetty. Setup is done via the artifactory admin window: defaults are: username: admin password: password (must be changed )
          Hide
          Gerrit Jansen van Vuuren added a comment -

          Hi,

          I've finally got around to doing writing the ant tasks needed to help deploy hive jars to maven.

          The maven upload is simpler for apache software foundation projects.
          The procedure for uploading is explained at http://www.apache.org/dev/release-publishing.html#repository-guide and this is what the ant tasks in the patch does.

          1. The patch changes ivy.xml files to include a info tag with licence information.
          2. uses a makepom task that will generate the pom.xml file for each sub project i.e. ql. shims etc.
          3. prepare the maven deploy
          a. by creating a build/maven directory,
          b. copying the jar files like hive_exec.jar and naming them hive-exec-$

          {version}

          .jar
          c. copying the pom.xml files
          d. copying over the apache 2 licence (hope this is the correct licence.
          e. creating the required checksums for all files in the build/maven directory

          So to use this and generate the contents for the build/maven directory that will contain jars, poms, and licences folder just run the prepare-maven-publish task.
          The final task is the maven-publish task that does a scp to people.apache.org:/www/people.apache.org/repo/m1-ibiblio-rsync-repository/org/apache/hadoop/hive/, if the poms,jars, and licences folders are in here it will automatically be synced with the main maven repo.

          I have some questions:
          -> Do you guys have access to the /www/people.apache.org/repo/m1-ibiblio-rsync-repository/org/apache/hadoop/hive/ ?
          -> Is the assumption correct that hive uses the APACHE 2.0 licence?
          -> Would somebody be able to apply this patch after review (not commit) but just to generate the jars,poms for the hive-4x release to deploy this to maven and then for the hive-5x release?

          Show
          Gerrit Jansen van Vuuren added a comment - Hi, I've finally got around to doing writing the ant tasks needed to help deploy hive jars to maven. The maven upload is simpler for apache software foundation projects. The procedure for uploading is explained at http://www.apache.org/dev/release-publishing.html#repository-guide and this is what the ant tasks in the patch does. 1. The patch changes ivy.xml files to include a info tag with licence information. 2. uses a makepom task that will generate the pom.xml file for each sub project i.e. ql. shims etc. 3. prepare the maven deploy a. by creating a build/maven directory, b. copying the jar files like hive_exec.jar and naming them hive-exec-$ {version} .jar c. copying the pom.xml files d. copying over the apache 2 licence (hope this is the correct licence. e. creating the required checksums for all files in the build/maven directory So to use this and generate the contents for the build/maven directory that will contain jars, poms, and licences folder just run the prepare-maven-publish task. The final task is the maven-publish task that does a scp to people.apache.org:/www/people.apache.org/repo/m1-ibiblio-rsync-repository/org/apache/hadoop/hive/, if the poms,jars, and licences folders are in here it will automatically be synced with the main maven repo. I have some questions: -> Do you guys have access to the /www/people.apache.org/repo/m1-ibiblio-rsync-repository/org/apache/hadoop/hive/ ? -> Is the assumption correct that hive uses the APACHE 2.0 licence? -> Would somebody be able to apply this patch after review (not commit) but just to generate the jars,poms for the hive-4x release to deploy this to maven and then for the hive-5x release?
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095-Sample.patch [ 12437608 ]
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095-0.4.1.patch [ 12437609 ]
          Hide
          Gerrit Jansen van Vuuren added a comment -

          Hi this patch is exactly the same as the others expect that its for the current hive trunk.
          It also used $

          {version}

          inside the ivy.xml files to be more generic.

          Show
          Gerrit Jansen van Vuuren added a comment - Hi this patch is exactly the same as the others expect that its for the current hive trunk. It also used $ {version} inside the ivy.xml files to be more generic.
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095-trunk.patch [ 12437616 ]
          Gerrit Jansen van Vuuren made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.6.0 [ 12314524 ]
          Fix Version/s 0.6.0 [ 12314524 ]
          Tags HIVE-1095-trunk.patch
          Hide
          He Yongqiang added a comment -

          >>Would somebody be able to apply this patch after review (not commit)
          Does this mean the patch need to regenerated after any conflicting changes?

          Show
          He Yongqiang added a comment - >>Would somebody be able to apply this patch after review (not commit) Does this mean the patch need to regenerated after any conflicting changes?
          Hide
          Gerrit Jansen van Vuuren added a comment -

          What I said was a bit confusing I've realized and what I meant was:

          There are two parts:

          -> Commiting (if reviewed and accepted) the HIVE-1095-trunk.patch to the trunk. This is generated against trunk.

          -> Using the HIVE-1095-0.4.1.patch against the version 0.4.1 of hive to generate the maven artifacts for 0.4.1 hive, are commits allowed for already versioned releases? if so then it would be better to have it committed cause any changes to build.xml, ivy.xml or build-common.xml would mean that the patch needs generation.

          So the broad scope and idea would be to publish the already released hive versions to the maven repo:
          0.3.0
          0.4.0
          0.4.1
          0.5.0
          and then have the build in trunk so that when another release is made the maven publishing code is already in the build and its only needed to run ant maven-publish.

          By writing this I've realized that I probably need to generate the patches for the builds on the other versions of hive also, should I do this and attach to this task?

          Show
          Gerrit Jansen van Vuuren added a comment - What I said was a bit confusing I've realized and what I meant was: There are two parts: -> Commiting (if reviewed and accepted) the HIVE-1095 -trunk.patch to the trunk. This is generated against trunk. -> Using the HIVE-1095 -0.4.1.patch against the version 0.4.1 of hive to generate the maven artifacts for 0.4.1 hive, are commits allowed for already versioned releases? if so then it would be better to have it committed cause any changes to build.xml, ivy.xml or build-common.xml would mean that the patch needs generation. So the broad scope and idea would be to publish the already released hive versions to the maven repo: 0.3.0 0.4.0 0.4.1 0.5.0 and then have the build in trunk so that when another release is made the maven publishing code is already in the build and its only needed to run ant maven-publish. By writing this I've realized that I probably need to generate the patches for the builds on the other versions of hive also, should I do this and attach to this task?
          Hide
          Gerrit Jansen van Vuuren added a comment -

          I might not be making sense but thinking again, should I just use this to generate the poms and jars for 0.3.0, 0.4.0, 0.4.1, and 0.5.0,
          i.e. have a tar.gz containing the poms, jars and licences folders for each version. Then attach those to this task. Then these can be scp(ed) over to the apache servers (I'll include a little ant build.xml to do this).

          Then we only have to worry about commit on the trunk. using Hive-1095-trunk.patch.

          Any thoughts?

          Show
          Gerrit Jansen van Vuuren added a comment - I might not be making sense but thinking again, should I just use this to generate the poms and jars for 0.3.0, 0.4.0, 0.4.1, and 0.5.0, i.e. have a tar.gz containing the poms, jars and licences folders for each version. Then attach those to this task. Then these can be scp(ed) over to the apache servers (I'll include a little ant build.xml to do this). Then we only have to worry about commit on the trunk. using Hive-1095-trunk.patch. Any thoughts?
          Hide
          Edward Capriolo added a comment -

          If it is a lot of effort making patches for 0.3.0 and 0.4.0 could be skipped. Generally we have been applying most changes to trunk and sometimes the latest branch.

          Show
          Edward Capriolo added a comment - If it is a lot of effort making patches for 0.3.0 and 0.4.0 could be skipped. Generally we have been applying most changes to trunk and sometimes the latest branch.
          Hide
          Gerrit Jansen van Vuuren added a comment -

          Yep would be easier.

          So then we have:
          -> Patch for trunk as Hive-1095-trunk.patch
          -> I'll generate the ant script to do the hive for verions 0.4.1 and 0.5

          How does that sound?

          Show
          Gerrit Jansen van Vuuren added a comment - Yep would be easier. So then we have: -> Patch for trunk as Hive-1095-trunk.patch -> I'll generate the ant script to do the hive for verions 0.4.1 and 0.5 How does that sound?
          Hide
          Gerrit Jansen van Vuuren added a comment -

          Making new patch release

          Show
          Gerrit Jansen van Vuuren added a comment - Making new patch release
          Gerrit Jansen van Vuuren made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Gerrit Jansen van Vuuren added a comment -

          This package is meant to be used to prepare the hive 0.4.1 and 0.5.0 jars for maven deploy as per http://www.apache.org/dev/release-publishing.html#repository-guide.

          The tar.gz contains:
          -> build.xml
          ->resources/
          ->0.5.0/
          ->poms, licences
          ->0.4.1/
          ->poms, licences

          How to use for hive 0.4.1:
          1)Extract the hiveReleasedToMaven.tar.gz to a directory.
          2)At command line type: ant preparejars-0.4.1
          3)On success the poms, jars, licences directories can be found at build/dist/0.4.1
          4)Use scp to deploy to [username]:[password]@people.apache.org:/www/people.apache.org/repo/m1-ibiblio-rsync-repository/org/apache/hadoop/hive/

          How to use for hive 0.5.0:
          1)Extract the hiveReleasedToMaven.tar.gz to a directory.
          2)At command line type: ant preparejars-0.5.0
          3)On success the poms,jars, licences directories can be found at build/dist/0.5.0
          4)Use scp to deploy to [username]:[password]@people.apache.org:/www/people.apache.org/repo/m1-ibiblio-rsync-repository/org/apache/hadoop/hive/

          Show
          Gerrit Jansen van Vuuren added a comment - This package is meant to be used to prepare the hive 0.4.1 and 0.5.0 jars for maven deploy as per http://www.apache.org/dev/release-publishing.html#repository-guide . The tar.gz contains: -> build.xml ->resources/ ->0.5.0/ ->poms, licences ->0.4.1/ ->poms, licences How to use for hive 0.4.1: 1)Extract the hiveReleasedToMaven.tar.gz to a directory. 2)At command line type: ant preparejars-0.4.1 3)On success the poms, jars, licences directories can be found at build/dist/0.4.1 4)Use scp to deploy to [username] : [password] @people.apache.org:/www/people.apache.org/repo/m1-ibiblio-rsync-repository/org/apache/hadoop/hive/ How to use for hive 0.5.0: 1)Extract the hiveReleasedToMaven.tar.gz to a directory. 2)At command line type: ant preparejars-0.5.0 3)On success the poms,jars, licences directories can be found at build/dist/0.5.0 4)Use scp to deploy to [username] : [password] @people.apache.org:/www/people.apache.org/repo/m1-ibiblio-rsync-repository/org/apache/hadoop/hive/
          Gerrit Jansen van Vuuren made changes -
          Attachment hiveReleasedToMaven.tar.gz [ 12438814 ]
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095-0.4.1.patch [ 12437609 ]
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095-Sample.patch [ 12437608 ]
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095-trunk.patch [ 12437616 ]
          Hide
          Gerrit Jansen van Vuuren added a comment -

          This patch is meant to be commited (if accepted) to the hive trunk.

          Purpose:
          Firstly to create a build/maven directory structure with the jar, autogenerated pom files and, licences (apache LICENCE-2.0).
          Optional deploy target for deploying to the maven Nexus repository Manager.

          How to create maven directory structure:
          1) Type in ant prepare-maven-publish
          The maven directory structure will be in build/maven/ containing poms,jars,licences folders.

          I'll write a comment on the different deployment strategies.

          Show
          Gerrit Jansen van Vuuren added a comment - This patch is meant to be commited (if accepted) to the hive trunk. Purpose: Firstly to create a build/maven directory structure with the jar, autogenerated pom files and, licences (apache LICENCE-2.0). Optional deploy target for deploying to the maven Nexus repository Manager. How to create maven directory structure: 1) Type in ant prepare-maven-publish The maven directory structure will be in build/maven/ containing poms,jars,licences folders. I'll write a comment on the different deployment strategies.
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095-trunk.patch [ 12438837 ]
          Hide
          Gerrit Jansen van Vuuren added a comment -

          Hi,

          I've been doing some research and browsing and have find 2 major ways in which ASF projects can do deployments for maven, scp or using the ASF Nexus Maven Repo Manager. Please ignore step 4 on the above patch for uploading versions 0.4.1 and 0.5.0.

          One option is using the scp to the http://people.apache.org/repo/m2-ibiblio-rsync-repository repository but this is not the most appropriate and from what I've read ASF projects should use the ASF Nexus Maven Repo Manager. I've looked at the hadoop core build.xml files and it uses this strategy.

          There are 2 ways of deploying using Nexus:
          1) Goto https://repository.apache.org/ and deploy the jar files manually see ( http://www.sonatype.com/books/nexus-book/reference/staging-sect-uploading-staged.html)
          2) Via maven -> Use the ant target provided in this patch called maven-publish

          Prior to doing either of these access credentials must be applied for, the https://issues.apache.org/jira/browse/INFRA-1896 tasks contains the information on how to go about this, with some reference documentation links.

          Show
          Gerrit Jansen van Vuuren added a comment - Hi, I've been doing some research and browsing and have find 2 major ways in which ASF projects can do deployments for maven, scp or using the ASF Nexus Maven Repo Manager. Please ignore step 4 on the above patch for uploading versions 0.4.1 and 0.5.0. One option is using the scp to the http://people.apache.org/repo/m2-ibiblio-rsync-repository repository but this is not the most appropriate and from what I've read ASF projects should use the ASF Nexus Maven Repo Manager. I've looked at the hadoop core build.xml files and it uses this strategy. There are 2 ways of deploying using Nexus: 1) Goto https://repository.apache.org/ and deploy the jar files manually see ( http://www.sonatype.com/books/nexus-book/reference/staging-sect-uploading-staged.html ) 2) Via maven -> Use the ant target provided in this patch called maven-publish Prior to doing either of these access credentials must be applied for, the https://issues.apache.org/jira/browse/INFRA-1896 tasks contains the information on how to go about this, with some reference documentation links.
          Gerrit Jansen van Vuuren made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          John Sichi made changes -
          Fix Version/s 0.6.0 [ 12314524 ]
          Hide
          Carl Steinbach added a comment -

          The patch applies cleanly, but I get the following error when I run the prepare-maven-publish target:

          
          % ant maven-publish-artifact
          Buildfile: /Users/carl/Projects/hive/build.xml
          
          ant-task-download:
                [get] Getting: http://repo2.maven.org/maven2/org/apache/maven/maven-ant-tasks/2.1.0/maven-ant-tasks-2.1.0.jar
                [get] To: /Users/carl/Projects/hive/build/maven-ant-tasks-2.1.0.jar
          
          mvn-taskdef:
          
          maven-publish-artifact:
          [artifact:pom] An error has occurred while processing the Maven artifact tasks.
          [artifact:pom]  Diagnosis:
          [artifact:pom] 
          [artifact:pom] Unable to initialize POM hive-${hive.project}-0.6.0.pom: Could not find the model file 
          '/Users/carl/Projects/hive/build/maven/poms/hive-${hive.project}-0.6.0.pom'. for project unknown
          [artifact:pom] /Users/carl/Projects/hive/build/maven/poms/hive-${hive.project}-0.6.0.pom (No such file or directory)
          
          BUILD FAILED
          /Users/carl/Projects/hive/build.xml:410: Unable to initialize POM hive-${hive.project}-0.6.0.pom:
          Could not find the model file '/Users/carl/Projects/hive/build/maven/poms/hive-${hive.project}-0.6.0.pom'. for project unknown
          
          Total time: 5 seconds
          

          It looks like the make-pom target needs to depend on ivy-init?

          Show
          Carl Steinbach added a comment - The patch applies cleanly, but I get the following error when I run the prepare-maven-publish target: % ant maven-publish-artifact Buildfile: /Users/carl/Projects/hive/build.xml ant-task-download: [get] Getting: http: //repo2.maven.org/maven2/org/apache/maven/maven-ant-tasks/2.1.0/maven-ant-tasks-2.1.0.jar [get] To: /Users/carl/Projects/hive/build/maven-ant-tasks-2.1.0.jar mvn-taskdef: maven-publish-artifact: [artifact:pom] An error has occurred while processing the Maven artifact tasks. [artifact:pom] Diagnosis: [artifact:pom] [artifact:pom] Unable to initialize POM hive-${hive.project}-0.6.0.pom: Could not find the model file '/Users/carl/Projects/hive/build/maven/poms/hive-${hive.project}-0.6.0.pom'. for project unknown [artifact:pom] /Users/carl/Projects/hive/build/maven/poms/hive-${hive.project}-0.6.0.pom (No such file or directory) BUILD FAILED /Users/carl/Projects/hive/build.xml:410: Unable to initialize POM hive-${hive.project}-0.6.0.pom: Could not find the model file '/Users/carl/Projects/hive/build/maven/poms/hive-${hive.project}-0.6.0.pom'. for project unknown Total time: 5 seconds It looks like the make-pom target needs to depend on ivy-init?
          Carl Steinbach made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Carl Steinbach made changes -
          Link This issue is blocked by HIVE-1817 [ HIVE-1817 ]
          Hide
          Andreas Neumann added a comment -

          Hi, I am working on Oozie (github.com/yahoo/oozie) and there is high demand for Hive support for Oozie. Currently that is problematic because there is no Hive release in the Apache Maven repository. Because Oozie is mavenized, we would need Maven artifacts available, preferably in the Apache repository. Any chance this could be resolved soon? Thanks -Andreas.

          Show
          Andreas Neumann added a comment - Hi, I am working on Oozie (github.com/yahoo/oozie) and there is high demand for Hive support for Oozie. Currently that is problematic because there is no Hive release in the Apache Maven repository. Because Oozie is mavenized, we would need Maven artifacts available, preferably in the Apache repository. Any chance this could be resolved soon? Thanks -Andreas.
          Show
          Carl Steinbach added a comment - Relevant ASF documentation: http://www.apache.org/dev/repository-faq.html http://www.apache.org/dev/publishing-maven-artifacts.html
          Hide
          Amareshwari Sriramadasu added a comment -

          Gerrit, are you planning to regenerate the patch incorporating Carl's comments?

          Show
          Amareshwari Sriramadasu added a comment - Gerrit, are you planning to regenerate the patch incorporating Carl's comments?
          Hide
          Gerrit Jansen van Vuuren added a comment -

          hi,
          I haven't looked at this patch for some time.
          I'll try and do this over the weekend. See how it goes.

          Show
          Gerrit Jansen van Vuuren added a comment - hi, I haven't looked at this patch for some time. I'll try and do this over the weekend. See how it goes.
          Hide
          Amareshwari Sriramadasu added a comment -

          Gerrit, did you get a chance to do this? As it is little urgent for us, I would like to take this up if you can not find time for finishing it.

          Show
          Amareshwari Sriramadasu added a comment - Gerrit, did you get a chance to do this? As it is little urgent for us, I would like to take this up if you can not find time for finishing it.
          Hide
          Gerrit Jansen van Vuuren added a comment -

          hive patch for generating maven artifacts for hive.

          Show
          Gerrit Jansen van Vuuren added a comment - hive patch for generating maven artifacts for hive.
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095.v2.PATCH [ 12473694 ]
          Hide
          Gerrit Jansen van Vuuren added a comment -

          Hi,

          The above patch compiles and generates the maven artifacts to : build/maven
          3 directories are generated: jars, licences, and poms.

          Notes:
          The poms are automatically generated from the ivy dependencies.
          md5 and sha1 checksums is generated for each artifact (jar, pom, and licence).

          Target to run:
          From the trunk/build.xml the target make-maven can be used.
          From each sub project the target make-pom can be used.

          As far as I could read this should be enough to deploy to a maven repository.
          The groupId would be org.apache.hadoop.hive, this is what was in the ivy files.

          I have added another target called maven-publish in the trunk/build.xml
          This has not been tested and is supposed to make deployment to a maven repository easier.

          Show
          Gerrit Jansen van Vuuren added a comment - Hi, The above patch compiles and generates the maven artifacts to : build/maven 3 directories are generated: jars, licences, and poms. Notes: The poms are automatically generated from the ivy dependencies. md5 and sha1 checksums is generated for each artifact (jar, pom, and licence). Target to run: From the trunk/build.xml the target make-maven can be used. From each sub project the target make-pom can be used. As far as I could read this should be enough to deploy to a maven repository. The groupId would be org.apache.hadoop.hive, this is what was in the ivy files. I have added another target called maven-publish in the trunk/build.xml This has not been tested and is supposed to make deployment to a maven repository easier.
          Hide
          Carl Steinbach added a comment -

          Since Hive is now a TLP I think the groupId should be "org.apache.hive"

          Show
          Carl Steinbach added a comment - Since Hive is now a TLP I think the groupId should be "org.apache.hive"
          Hide
          Gerrit Jansen van Vuuren added a comment -

          True it should be org.apache.hive
          here is the updated patch.

          (I also applied some space formatting to the build.xml file).

          Show
          Gerrit Jansen van Vuuren added a comment - True it should be org.apache.hive here is the updated patch. (I also applied some space formatting to the build.xml file).
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095.v3.PATCH [ 12473728 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          Also, homeepage should be http://hive.apache.org, not http//hadoop.apache.org/hive.

          Show
          Amareshwari Sriramadasu added a comment - Also, homeepage should be http://hive.apache.org , not http//hadoop.apache.org/hive.
          Hide
          Giridharan Kesavan added a comment -

          v3 version of patch fails with a conflict on build-common.xml

          Hunk #1 FAILED at 1.
          1 out of 1 hunk FAILED – saving rejects to file build-common.xml.rej

          Show
          Giridharan Kesavan added a comment - v3 version of patch fails with a conflict on build-common.xml Hunk #1 FAILED at 1. 1 out of 1 hunk FAILED – saving rejects to file build-common.xml.rej
          Hide
          Gerrit Jansen van Vuuren added a comment -

          fixed,

          Show
          Gerrit Jansen van Vuuren added a comment - fixed,
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095.v4.PATCH [ 12473797 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          When I ran make-maven target, build failed with following error:

          BUILD FAILED
          /Users/amarsri/hive/build.xml:706: Warning: Could not find file /Users/amarsri/hive/build/ql/pom.xml to copy.
          

          I don't see any pom files created in build directory.

          Show
          Amareshwari Sriramadasu added a comment - When I ran make-maven target, build failed with following error: BUILD FAILED /Users/amarsri/hive/build.xml:706: Warning: Could not find file /Users/amarsri/hive/build/ql/pom.xml to copy. I don't see any pom files created in build directory.
          Hide
          Giridharan Kesavan added a comment -

          I got the same error as Amareshwari mentioned in the previous comment. But I fixed locally by changing the dependency of make-pom target. ie made the make-pom target to depend on check-ivy instead of ivy-init.

          build-common.xml 
          - <target name="make-pom" if="ivy.present" depends="ivy-init, jar">
          + <target name="make-pom" if="ivy.present" depends="check-ivy, jar">
          

          This seem to work fine..

          About snapshot versioning:
          I think we cannot publish snapshots to the staging/release repository. Snapshots can only go the the snapshots repo.
          We need to fix the version string so that we can publish to the staging/release repo.

          Show
          Giridharan Kesavan added a comment - I got the same error as Amareshwari mentioned in the previous comment. But I fixed locally by changing the dependency of make-pom target. ie made the make-pom target to depend on check-ivy instead of ivy-init. build-common.xml - <target name= "make-pom" if= "ivy.present" depends= "ivy-init, jar" > + <target name= "make-pom" if= "ivy.present" depends= "check-ivy, jar" > This seem to work fine.. About snapshot versioning: I think we cannot publish snapshots to the staging/release repository. Snapshots can only go the the snapshots repo. We need to fix the version string so that we can publish to the staging/release repo.
          Hide
          Giridharan Kesavan added a comment -

          someone with hive committer access should try this to see if we are able to publish to the nexus staging repo.

          ant make-maven -Dversion=0.8.0
          ant maven-publish -Dversion=0.8.0

          Show
          Giridharan Kesavan added a comment - someone with hive committer access should try this to see if we are able to publish to the nexus staging repo. ant make-maven -Dversion=0.8.0 ant maven-publish -Dversion=0.8.0
          Hide
          Carl Steinbach added a comment -

          @Giridharan: 0.8.0 is the current trunk development branch. We haven't voted to release this version yet, so we shouldn't publish anything other than 0.8.0 SNAPSHOTs to the snapshot repository. If you want to publish artifacts to the release repo then we need to backport this patch to branch-0.7.

          Show
          Carl Steinbach added a comment - @Giridharan: 0.8.0 is the current trunk development branch. We haven't voted to release this version yet, so we shouldn't publish anything other than 0.8.0 SNAPSHOTs to the snapshot repository. If you want to publish artifacts to the release repo then we need to backport this patch to branch-0.7.
          Hide
          Gerrit Jansen van Vuuren added a comment -

          patch with fix check-ivy

          to make the pom files and jars type make-maven

          Show
          Gerrit Jansen van Vuuren added a comment - patch with fix check-ivy to make the pom files and jars type make-maven
          Gerrit Jansen van Vuuren made changes -
          Attachment HIVE-1095.v5.PATCH [ 12474700 ]
          Hide
          Gerrit Jansen van Vuuren added a comment -

          Thanks Giridharan for the fix proposed above.
          I've added that to the v5 patch file.

          Show
          Gerrit Jansen van Vuuren added a comment - Thanks Giridharan for the fix proposed above. I've added that to the v5 patch file.
          Hide
          Giridharan Kesavan added a comment -

          @Carl Steinbach. I proposed this for the purpose of testing; For publishing to staging repo we don't have to go for a voting. If someone can confirm that this patch work by publishing artifacts to staging then the same committer can drop the artifact from the staging repo. And later someone can work on porting the patch to 0.7 branch.

          Show
          Giridharan Kesavan added a comment - @Carl Steinbach. I proposed this for the purpose of testing; For publishing to staging repo we don't have to go for a voting. If someone can confirm that this patch work by publishing artifacts to staging then the same committer can drop the artifact from the staging repo. And later someone can work on porting the patch to 0.7 branch.
          Hide
          Amareshwari Sriramadasu added a comment -

          someone with hive committer access should try this to see if we are able to publish to the nexus staging repo.

          I tried that, but I'm getting 401 error, even with proper settings in ~/.m2/settings.xml.
          I tried to publish snapshot also by changing apache.releases.https to apache.snapshots.https in the patch and running the above for 0.8.0-SNAPSHOT version. It failed with error 401.

          Can someone else try this?

          Show
          Amareshwari Sriramadasu added a comment - someone with hive committer access should try this to see if we are able to publish to the nexus staging repo. I tried that, but I'm getting 401 error, even with proper settings in ~/.m2/settings.xml. I tried to publish snapshot also by changing apache.releases.https to apache.snapshots.https in the patch and running the above for 0.8.0-SNAPSHOT version. It failed with error 401. Can someone else try this?
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/543/
          -----------------------------------------------------------

          Review request for hive.

          Summary
          -------

          HIVE-1095: https://issues.apache.org/jira/secure/attachment/12474700/HIVE-1095.v5.PATCH

          This addresses bug HIVE-1095.
          https://issues.apache.org/jira/browse/HIVE-1095

          Diffs


          trunk/LICENSE-2.0.txt PRE-CREATION
          trunk/ant/ivy.xml 1082104
          trunk/build-common.xml 1082104
          trunk/build.xml 1082104
          trunk/cli/ivy.xml 1082104
          trunk/common/ivy.xml 1082104
          trunk/contrib/ivy.xml 1082104
          trunk/hbase-handler/ivy.xml 1082104
          trunk/hwi/ivy.xml 1082104
          trunk/ivy.xml 1082104
          trunk/metastore/ivy.xml 1082104
          trunk/ql/ivy.xml 1082104
          trunk/serde/ivy.xml 1082104
          trunk/shims/ivy.xml 1082104

          Diff: https://reviews.apache.org/r/543/diff

          Testing
          -------

          Thanks,

          Carl

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/543/ ----------------------------------------------------------- Review request for hive. Summary ------- HIVE-1095 : https://issues.apache.org/jira/secure/attachment/12474700/HIVE-1095.v5.PATCH This addresses bug HIVE-1095 . https://issues.apache.org/jira/browse/HIVE-1095 Diffs trunk/LICENSE-2.0.txt PRE-CREATION trunk/ant/ivy.xml 1082104 trunk/build-common.xml 1082104 trunk/build.xml 1082104 trunk/cli/ivy.xml 1082104 trunk/common/ivy.xml 1082104 trunk/contrib/ivy.xml 1082104 trunk/hbase-handler/ivy.xml 1082104 trunk/hwi/ivy.xml 1082104 trunk/ivy.xml 1082104 trunk/metastore/ivy.xml 1082104 trunk/ql/ivy.xml 1082104 trunk/serde/ivy.xml 1082104 trunk/shims/ivy.xml 1082104 Diff: https://reviews.apache.org/r/543/diff Testing ------- Thanks, Carl
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/543/#review384
          -----------------------------------------------------------

          trunk/build-common.xml
          <https://reviews.apache.org/r/543/#comment728>

          url should be http://svn.apache.org/repos/asf/hive.

          trunk/build.xml
          <https://reviews.apache.org/r/543/#comment726>

          Shall we add snapshot repo also here for https://repository.apache.org/content/repositories/snapshots ?

          trunk/build.xml
          <https://reviews.apache.org/r/543/#comment727>

          We should have target to publish snapshots also. Repository is apache.snapshots.https

          trunk/cli/ivy.xml
          <https://reviews.apache.org/r/543/#comment732>

          why is module name changed to serde?

          trunk/hbase-handler/ivy.xml
          <https://reviews.apache.org/r/543/#comment731>

          should module name be hbase-handler?

          trunk/ivy.xml
          <https://reviews.apache.org/r/543/#comment729>

          url should be http://hive.apache.org/

          trunk/ql/ivy.xml
          <https://reviews.apache.org/r/543/#comment730>

          why is module name changed to exec?

          • Amareshwari

          On 2011-04-01 20:12:49, Carl Steinbach wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/543/

          -----------------------------------------------------------

          (Updated 2011-04-01 20:12:49)

          Review request for hive.

          Summary

          -------

          HIVE-1095: https://issues.apache.org/jira/secure/attachment/12474700/HIVE-1095.v5.PATCH

          This addresses bug HIVE-1095.

          https://issues.apache.org/jira/browse/HIVE-1095

          Diffs

          -----

          trunk/LICENSE-2.0.txt PRE-CREATION

          trunk/ant/ivy.xml 1082104

          trunk/build-common.xml 1082104

          trunk/build.xml 1082104

          trunk/cli/ivy.xml 1082104

          trunk/common/ivy.xml 1082104

          trunk/contrib/ivy.xml 1082104

          trunk/hbase-handler/ivy.xml 1082104

          trunk/hwi/ivy.xml 1082104

          trunk/ivy.xml 1082104

          trunk/metastore/ivy.xml 1082104

          trunk/ql/ivy.xml 1082104

          trunk/serde/ivy.xml 1082104

          trunk/shims/ivy.xml 1082104

          Diff: https://reviews.apache.org/r/543/diff

          Testing

          -------

          Thanks,

          Carl

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/543/#review384 ----------------------------------------------------------- trunk/build-common.xml < https://reviews.apache.org/r/543/#comment728 > url should be http://svn.apache.org/repos/asf/hive . trunk/build.xml < https://reviews.apache.org/r/543/#comment726 > Shall we add snapshot repo also here for https://repository.apache.org/content/repositories/snapshots ? trunk/build.xml < https://reviews.apache.org/r/543/#comment727 > We should have target to publish snapshots also. Repository is apache.snapshots.https trunk/cli/ivy.xml < https://reviews.apache.org/r/543/#comment732 > why is module name changed to serde? trunk/hbase-handler/ivy.xml < https://reviews.apache.org/r/543/#comment731 > should module name be hbase-handler? trunk/ivy.xml < https://reviews.apache.org/r/543/#comment729 > url should be http://hive.apache.org/ trunk/ql/ivy.xml < https://reviews.apache.org/r/543/#comment730 > why is module name changed to exec? Amareshwari On 2011-04-01 20:12:49, Carl Steinbach wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/543/ ----------------------------------------------------------- (Updated 2011-04-01 20:12:49) Review request for hive. Summary ------- HIVE-1095 : https://issues.apache.org/jira/secure/attachment/12474700/HIVE-1095.v5.PATCH This addresses bug HIVE-1095 . https://issues.apache.org/jira/browse/HIVE-1095 Diffs ----- trunk/LICENSE-2.0.txt PRE-CREATION trunk/ant/ivy.xml 1082104 trunk/build-common.xml 1082104 trunk/build.xml 1082104 trunk/cli/ivy.xml 1082104 trunk/common/ivy.xml 1082104 trunk/contrib/ivy.xml 1082104 trunk/hbase-handler/ivy.xml 1082104 trunk/hwi/ivy.xml 1082104 trunk/ivy.xml 1082104 trunk/metastore/ivy.xml 1082104 trunk/ql/ivy.xml 1082104 trunk/serde/ivy.xml 1082104 trunk/shims/ivy.xml 1082104 Diff: https://reviews.apache.org/r/543/diff Testing ------- Thanks, Carl
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/543/#review385
          -----------------------------------------------------------

          trunk/build.xml
          <https://reviews.apache.org/r/543/#comment733>

          Shouldn't we copy contrib, jdbc, hbase-handler, hwi and service jars also?

          • Amareshwari

          On 2011-04-01 20:12:49, Carl Steinbach wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/543/

          -----------------------------------------------------------

          (Updated 2011-04-01 20:12:49)

          Review request for hive.

          Summary

          -------

          HIVE-1095: https://issues.apache.org/jira/secure/attachment/12474700/HIVE-1095.v5.PATCH

          This addresses bug HIVE-1095.

          https://issues.apache.org/jira/browse/HIVE-1095

          Diffs

          -----

          trunk/LICENSE-2.0.txt PRE-CREATION

          trunk/ant/ivy.xml 1082104

          trunk/build-common.xml 1082104

          trunk/build.xml 1082104

          trunk/cli/ivy.xml 1082104

          trunk/common/ivy.xml 1082104

          trunk/contrib/ivy.xml 1082104

          trunk/hbase-handler/ivy.xml 1082104

          trunk/hwi/ivy.xml 1082104

          trunk/ivy.xml 1082104

          trunk/metastore/ivy.xml 1082104

          trunk/ql/ivy.xml 1082104

          trunk/serde/ivy.xml 1082104

          trunk/shims/ivy.xml 1082104

          Diff: https://reviews.apache.org/r/543/diff

          Testing

          -------

          Thanks,

          Carl

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/543/#review385 ----------------------------------------------------------- trunk/build.xml < https://reviews.apache.org/r/543/#comment733 > Shouldn't we copy contrib, jdbc, hbase-handler, hwi and service jars also? Amareshwari On 2011-04-01 20:12:49, Carl Steinbach wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/543/ ----------------------------------------------------------- (Updated 2011-04-01 20:12:49) Review request for hive. Summary ------- HIVE-1095 : https://issues.apache.org/jira/secure/attachment/12474700/HIVE-1095.v5.PATCH This addresses bug HIVE-1095 . https://issues.apache.org/jira/browse/HIVE-1095 Diffs ----- trunk/LICENSE-2.0.txt PRE-CREATION trunk/ant/ivy.xml 1082104 trunk/build-common.xml 1082104 trunk/build.xml 1082104 trunk/cli/ivy.xml 1082104 trunk/common/ivy.xml 1082104 trunk/contrib/ivy.xml 1082104 trunk/hbase-handler/ivy.xml 1082104 trunk/hwi/ivy.xml 1082104 trunk/ivy.xml 1082104 trunk/metastore/ivy.xml 1082104 trunk/ql/ivy.xml 1082104 trunk/serde/ivy.xml 1082104 trunk/shims/ivy.xml 1082104 Diff: https://reviews.apache.org/r/543/diff Testing ------- Thanks, Carl
          Hide
          Amareshwari Sriramadasu added a comment -

          Thanks Carl for review request. Update the review board with some comments.

          Current patch is building the jars properly. Can someone try the commands put by Giri? I'm getting 401 error when i tried.

          Show
          Amareshwari Sriramadasu added a comment - Thanks Carl for review request. Update the review board with some comments. Current patch is building the jars properly. Can someone try the commands put by Giri? I'm getting 401 error when i tried.
          Hide
          Ning Zhang added a comment -

          I tried the first command: ant make-maven -Dversion=0.8.0-SNAPSHOT -logfile make-maven.log
          and it seems succeeded. I'll attached make-maven.log. It would be nice that someone has the knowledge can take a look and see if it is correct.

          I haven't not run the other command to publish maven yet. I can run that as long as I get a +1 from committers who has the knowledge.

          Show
          Ning Zhang added a comment - I tried the first command: ant make-maven -Dversion=0.8.0-SNAPSHOT -logfile make-maven.log and it seems succeeded. I'll attached make-maven.log. It would be nice that someone has the knowledge can take a look and see if it is correct. I haven't not run the other command to publish maven yet. I can run that as long as I get a +1 from committers who has the knowledge.
          Ning Zhang made changes -
          Attachment make-maven.log [ 12475533 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          I tried the first command: ant make-maven -Dversion=0.8.0-SNAPSHOT -logfile make-maven.log and it seems succeeded.

          It succeeded for me too, where as maven-publish failed with 401/authorization errors.

          It would be nice that someone has the knowledge can take a look and see if it is correct.

          Giri/Carl, can you help here?

          Show
          Amareshwari Sriramadasu added a comment - I tried the first command: ant make-maven -Dversion=0.8.0-SNAPSHOT -logfile make-maven.log and it seems succeeded. It succeeded for me too, where as maven-publish failed with 401/authorization errors. It would be nice that someone has the knowledge can take a look and see if it is correct. Giri/Carl, can you help here?
          Hide
          Carl Steinbach added a comment -

          I'm making the changes that were flagged by Amareshwari, along with some others that I found, and will try running through the publication steps when I finish.

          Show
          Carl Steinbach added a comment - I'm making the changes that were flagged by Amareshwari, along with some others that I found, and will try running through the publication steps when I finish.
          Hide
          Ning Zhang added a comment -

          @Amareshwari, I got the same 401 error while running ant maven-publish.

          Show
          Ning Zhang added a comment - @Amareshwari, I got the same 401 error while running ant maven-publish.
          Hide
          Giridharan Kesavan added a comment -

          since everyone is getting a 401 error (I tried with the hudson CI user as well and got the same 401). We need to check with the nexus admin to see if hive repo is available and ready for artifact publishing.

          Show
          Giridharan Kesavan added a comment - since everyone is getting a 401 error (I tried with the hudson CI user as well and got the same 401). We need to check with the nexus admin to see if hive repo is available and ready for artifact publishing.
          Hide
          Amareshwari Sriramadasu added a comment -

          I created an INFRA ticket for nexus access: https://issues.apache.org/jira/browse/INFRA-3567.

          Show
          Amareshwari Sriramadasu added a comment - I created an INFRA ticket for nexus access: https://issues.apache.org/jira/browse/INFRA-3567 .
          Amareshwari Sriramadasu made changes -
          Link This issue is blocked by INFRA-3567 [ INFRA-3567 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          Now that INFRA ticket is resolved, I tried maven-publish for version 0.8.0-SNAPSHOT to repository apache.snapshots.https with repo url https://repository.apache.org/content/repositories/snapshots. I don't see 401 error any more.
          Could upload exec, serde, common, metastore jars successfully. You can see the uploaded jars at https://repository.apache.org/content/repositories/snapshots/org/apache/hive/
          But, shims jar could not be uploaded as version number is not picked up properly, it fails with following error:

          maven-publish-artifact:
          [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-http:jar:1.0-beta-2:runtime
          [artifact:deploy] Deploying to https://repository.apache.org/content/repositories/snapshots
          [artifact:deploy] Uploading: org/apache/hive/shims/working@mymachine/shims-working@mymachine.jar to repository apache.snapshots.https at https://repository.apache.org/content/repositories/snapshots
          [artifact:deploy] Transferring 70K from apache.snapshots.https
          [artifact:deploy] An error has occurred while processing the Maven artifact tasks.
          [artifact:deploy]  Diagnosis:
          [artifact:deploy] 
          [artifact:deploy] Error deploying artifact 'org.apache.hive:shims:jar': Error deploying artifact: Failed to transfer file: https://repository.apache.org/content/repositories/snapshots/org/apache/hive/shims/working@mymachine/shims-working@mymachine.jar. Return code is: 400
          

          Here is successful upload console output:

          maven-publish-artifact:
          [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-http:jar:1.0-beta-2:runtime
          [artifact:deploy] Deploying to https://repository.apache.org/content/repositories/snapshots
          [artifact:deploy] [INFO] Retrieving previous build number from apache.snapshots.https
          [artifact:deploy] [INFO] repository metadata for: 'snapshot org.apache.hive:metastore:0.8.0-SNAPSHOT' could not be found on repository: apache.snapshots.https, so will be created
          [artifact:deploy] Uploading: org/apache/hive/metastore/0.8.0-SNAPSHOT/metastore-0.8.0-20110419.165258-1.jar to repository apache.snapshots.https at https://repository.apache.org/content/repositories/snapshots
          [artifact:deploy] Transferring 1524K from apache.snapshots.https
          [artifact:deploy] Uploaded 1524K
          [artifact:deploy] [INFO] Uploading project information for metastore 0.8.0-20110419.165258-1
          [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https
          [artifact:deploy] [INFO] repository metadata for: 'snapshot org.apache.hive:metastore:0.8.0-SNAPSHOT' could not be found on repository: apache.snapshots.https, so will be created
          [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot org.apache.hive:metastore:0.8.0-SNAPSHOT'
          [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https
          [artifact:deploy] [INFO] repository metadata for: 'artifact org.apache.hive:metastore' could not be found on repository: apache.snapshots.https, so will be created
          [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact org.apache.hive:metastore'
          

          Giri, any idea why shims jar is not picked up properly?
          Carl, can you try with the patch that you made changes?

          Show
          Amareshwari Sriramadasu added a comment - Now that INFRA ticket is resolved, I tried maven-publish for version 0.8.0-SNAPSHOT to repository apache.snapshots.https with repo url https://repository.apache.org/content/repositories/snapshots . I don't see 401 error any more. Could upload exec, serde, common, metastore jars successfully. You can see the uploaded jars at https://repository.apache.org/content/repositories/snapshots/org/apache/hive/ But, shims jar could not be uploaded as version number is not picked up properly, it fails with following error: maven-publish-artifact: [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-http:jar:1.0-beta-2:runtime [artifact:deploy] Deploying to https://repository.apache.org/content/repositories/snapshots [artifact:deploy] Uploading: org/apache/hive/shims/working@mymachine/shims-working@mymachine.jar to repository apache.snapshots.https at https://repository.apache.org/content/repositories/snapshots [artifact:deploy] Transferring 70K from apache.snapshots.https [artifact:deploy] An error has occurred while processing the Maven artifact tasks. [artifact:deploy] Diagnosis: [artifact:deploy] [artifact:deploy] Error deploying artifact 'org.apache.hive:shims:jar': Error deploying artifact: Failed to transfer file: https://repository.apache.org/content/repositories/snapshots/org/apache/hive/shims/working@mymachine/shims-working@mymachine.jar. Return code is: 400 Here is successful upload console output: maven-publish-artifact: [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-http:jar:1.0-beta-2:runtime [artifact:deploy] Deploying to https://repository.apache.org/content/repositories/snapshots [artifact:deploy] [INFO] Retrieving previous build number from apache.snapshots.https [artifact:deploy] [INFO] repository metadata for: 'snapshot org.apache.hive:metastore:0.8.0-SNAPSHOT' could not be found on repository: apache.snapshots.https, so will be created [artifact:deploy] Uploading: org/apache/hive/metastore/0.8.0-SNAPSHOT/metastore-0.8.0-20110419.165258-1.jar to repository apache.snapshots.https at https://repository.apache.org/content/repositories/snapshots [artifact:deploy] Transferring 1524K from apache.snapshots.https [artifact:deploy] Uploaded 1524K [artifact:deploy] [INFO] Uploading project information for metastore 0.8.0-20110419.165258-1 [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] repository metadata for: 'snapshot org.apache.hive:metastore:0.8.0-SNAPSHOT' could not be found on repository: apache.snapshots.https, so will be created [artifact:deploy] [INFO] Uploading repository metadata for: 'snapshot org.apache.hive:metastore:0.8.0-SNAPSHOT' [artifact:deploy] [INFO] Retrieving previous metadata from apache.snapshots.https [artifact:deploy] [INFO] repository metadata for: 'artifact org.apache.hive:metastore' could not be found on repository: apache.snapshots.https, so will be created [artifact:deploy] [INFO] Uploading repository metadata for: 'artifact org.apache.hive:metastore' Giri, any idea why shims jar is not picked up properly? Carl, can you try with the patch that you made changes?
          Hide
          Amareshwari Sriramadasu added a comment -

          Giri, any idea why shims jar is not picked up properly?

          Got it. revision is missing in shims/ivy.xml file. After adding revision="$

          {version}" to info, shims jar is also submitted correctly. Please see uploaded jars at https://repository.apache.org/content/repositories/snapshots/org/apache/hive/.

          bq.I'm making the changes that were flagged by Amareshwari, along with some others that I found.
          Carl/Gerrit, Are you planning to upload a patch with changes suggested on review board? Can you add revision="${version}

          " to ivy.xml files in your new patch?

          Show
          Amareshwari Sriramadasu added a comment - Giri, any idea why shims jar is not picked up properly? Got it. revision is missing in shims/ivy.xml file. After adding revision="$ {version}" to info, shims jar is also submitted correctly. Please see uploaded jars at https://repository.apache.org/content/repositories/snapshots/org/apache/hive/ . bq.I'm making the changes that were flagged by Amareshwari, along with some others that I found. Carl/Gerrit, Are you planning to upload a patch with changes suggested on review board? Can you add revision="${version} " to ivy.xml files in your new patch?
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch incorporates all the comments on review board.
          Publishes all hive*.jar files to maven. See published snapshot jars in the repo at https://repository.apache.org/content/repositories/snapshots/org/apache/hive/.
          Carl/Gerrit, can you have a look at the patch?

          Show
          Amareshwari Sriramadasu added a comment - Patch incorporates all the comments on review board. Publishes all hive*.jar files to maven. See published snapshot jars in the repo at https://repository.apache.org/content/repositories/snapshots/org/apache/hive/ . Carl/Gerrit, can you have a look at the patch?
          Amareshwari Sriramadasu made changes -
          Attachment HIVE-1095.v6.patch [ 12476904 ]
          Amareshwari Sriramadasu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Tags HIVE-1095-trunk.patch
          Amareshwari Sriramadasu made changes -
          Fix Version/s 0.7.1 [ 12316336 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          I'm making the changes that were flagged by Amareshwari, along with some others that I found, and will try running through the publication steps when I finish.

          Carl, did you get chance to update the patch or to look at the patch i uploaded?

          Show
          Amareshwari Sriramadasu added a comment - I'm making the changes that were flagged by Amareshwari, along with some others that I found, and will try running through the publication steps when I finish. Carl, did you get chance to update the patch or to look at the patch i uploaded?
          Hide
          Carl Steinbach added a comment -

          @Amareshwari: I looked at the patch and noticed a couple things that should be fixed (e.g. there's already a copy of the ASF license in "LICENSE", so no need to add LICENSE-2.0.txt). If you post the patch to reviewboard I will add comments, or I can do the updates myself in the next couple of days.

          Show
          Carl Steinbach added a comment - @Amareshwari: I looked at the patch and noticed a couple things that should be fixed (e.g. there's already a copy of the ASF license in "LICENSE", so no need to add LICENSE-2.0.txt). If you post the patch to reviewboard I will add comments, or I can do the updates myself in the next couple of days.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/740/
          -----------------------------------------------------------

          Review request for hive and Carl Steinbach.

          Summary
          -------

          This addresses bug HIVE-1095.
          https://issues.apache.org/jira/browse/HIVE-1095

          Diffs


          trunk/LICENSE-2.0.txt PRE-CREATION
          trunk/ant/ivy.xml 1095392
          trunk/build-common.xml 1095392
          trunk/build.xml 1095392
          trunk/cli/ivy.xml 1095392
          trunk/common/ivy.xml 1095392
          trunk/contrib/ivy.xml 1095392
          trunk/hbase-handler/ivy.xml 1095392
          trunk/hwi/ivy.xml 1095392
          trunk/ivy.xml 1095392
          trunk/jdbc/ivy.xml PRE-CREATION
          trunk/metastore/ivy.xml 1095392
          trunk/ql/ivy.xml 1095392
          trunk/serde/ivy.xml 1095392
          trunk/service/ivy.xml PRE-CREATION
          trunk/shims/ivy.xml 1095392

          Diff: https://reviews.apache.org/r/740/diff

          Testing
          -------

          Thanks,

          Amareshwari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/740/ ----------------------------------------------------------- Review request for hive and Carl Steinbach. Summary ------- This addresses bug HIVE-1095 . https://issues.apache.org/jira/browse/HIVE-1095 Diffs trunk/LICENSE-2.0.txt PRE-CREATION trunk/ant/ivy.xml 1095392 trunk/build-common.xml 1095392 trunk/build.xml 1095392 trunk/cli/ivy.xml 1095392 trunk/common/ivy.xml 1095392 trunk/contrib/ivy.xml 1095392 trunk/hbase-handler/ivy.xml 1095392 trunk/hwi/ivy.xml 1095392 trunk/ivy.xml 1095392 trunk/jdbc/ivy.xml PRE-CREATION trunk/metastore/ivy.xml 1095392 trunk/ql/ivy.xml 1095392 trunk/serde/ivy.xml 1095392 trunk/service/ivy.xml PRE-CREATION trunk/shims/ivy.xml 1095392 Diff: https://reviews.apache.org/r/740/diff Testing ------- Thanks, Amareshwari
          Hide
          Amareshwari Sriramadasu added a comment -

          I could not update your last review request. Uploaded the last patch to a new review request. Can you please post comments there?

          Show
          Amareshwari Sriramadasu added a comment - I could not update your last review request. Uploaded the last patch to a new review request. Can you please post comments there?
          Hide
          Carl Steinbach added a comment -

          HIVE-1095.7.patch.txt:

          • Remove LICENSE-2.0.txt
          • Use Ivy to resolve maven-ant-tasks dependency.
          • Move some of the new maven targets into build-common.xml
          • Change the names of the user visible maven targets to "maven-build" and "maven-publish", and add descriptions for both.
          • Fixed the module fields in each of the */ivy.xml files.

          I verified that I can publish snapshots to the asf repo with this patch in place.

          Show
          Carl Steinbach added a comment - HIVE-1095 .7.patch.txt: Remove LICENSE-2.0.txt Use Ivy to resolve maven-ant-tasks dependency. Move some of the new maven targets into build-common.xml Change the names of the user visible maven targets to "maven-build" and "maven-publish", and add descriptions for both. Fixed the module fields in each of the */ivy.xml files. I verified that I can publish snapshots to the asf repo with this patch in place.
          Carl Steinbach made changes -
          Attachment HIVE-1095.7.patch.txt [ 12479392 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          Thanks Carl. Will take a look at the patch.

          Show
          Amareshwari Sriramadasu added a comment - Thanks Carl. Will take a look at the patch.
          Hide
          Amareshwari Sriramadasu added a comment -

          Changes look good to me.
          The same patch applies to branch 0.7 also and I could publish the snapshot jars for both 0.8.0 and 0.7.1
          Will commit the patch to trunk and branch 0.7 tomorrow, if no one objects.

          Show
          Amareshwari Sriramadasu added a comment - Changes look good to me. The same patch applies to branch 0.7 also and I could publish the snapshot jars for both 0.8.0 and 0.7.1 Will commit the patch to trunk and branch 0.7 tomorrow, if no one objects.
          Hide
          Carl Steinbach added a comment -

          +1

          Show
          Carl Steinbach added a comment - +1
          Hide
          Amareshwari Sriramadasu added a comment -

          I just committed this to trunk and branch 0.7.

          Thanks Gerrit and Carl !

          Show
          Amareshwari Sriramadasu added a comment - I just committed this to trunk and branch 0.7. Thanks Gerrit and Carl !
          Amareshwari Sriramadasu made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.8.0 [ 12316178 ]
          Resolution Fixed [ 1 ]
          Hide
          Gerrit Jansen van Vuuren added a comment -

          Great!!!!

          Thanks Carl, Amareshwari for seeing this through.

          Show
          Gerrit Jansen van Vuuren added a comment - Great!!!! Thanks Carl, Amareshwari for seeing this through.
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #736 (See https://builds.apache.org/hudson/job/Hive-trunk-h0.21/736/)
          HIVE-1095. Hive in Maven. Contributed by Gerrit Jansen van Vuuren, Amareshwari Sriramadasu and Carl Steinbach.

          amareshwari : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1124164
          Files :

          • /hive/trunk/ant/ivy.xml
          • /hive/trunk/ivy.xml
          • /hive/trunk/jdbc/ivy.xml
          • /hive/trunk/ql/ivy.xml
          • /hive/trunk/build.xml
          • /hive/trunk/service/ivy.xml
          • /hive/trunk/hbase-handler/ivy.xml
          • /hive/trunk/contrib/ivy.xml
          • /hive/trunk/shims/ivy.xml
          • /hive/trunk/hwi/ivy.xml
          • /hive/trunk/ivy/libraries.properties
          • /hive/trunk/metastore/ivy.xml
          • /hive/trunk/cli/ivy.xml
          • /hive/trunk/serde/ivy.xml
          • /hive/trunk/common/ivy.xml
          • /hive/trunk/build-common.xml
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #736 (See https://builds.apache.org/hudson/job/Hive-trunk-h0.21/736/ ) HIVE-1095 . Hive in Maven. Contributed by Gerrit Jansen van Vuuren, Amareshwari Sriramadasu and Carl Steinbach. amareshwari : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1124164 Files : /hive/trunk/ant/ivy.xml /hive/trunk/ivy.xml /hive/trunk/jdbc/ivy.xml /hive/trunk/ql/ivy.xml /hive/trunk/build.xml /hive/trunk/service/ivy.xml /hive/trunk/hbase-handler/ivy.xml /hive/trunk/contrib/ivy.xml /hive/trunk/shims/ivy.xml /hive/trunk/hwi/ivy.xml /hive/trunk/ivy/libraries.properties /hive/trunk/metastore/ivy.xml /hive/trunk/cli/ivy.xml /hive/trunk/serde/ivy.xml /hive/trunk/common/ivy.xml /hive/trunk/build-common.xml
          Carl Steinbach made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          105d 7h 59m 2 Carl Steinbach 17/Jun/10 22:45
          Open Open Patch Available Patch Available
          344d 23h 23m 3 Amareshwari Sriramadasu 20/Apr/11 17:24
          Patch Available Patch Available Resolved Resolved
          27d 18h 1 Amareshwari Sriramadasu 18/May/11 11:25
          Resolved Resolved Closed Closed
          69d 17h 54m 1 Carl Steinbach 27/Jul/11 05:20

            People

            • Assignee:
              Unassigned
              Reporter:
              Gerrit Jansen van Vuuren
            • Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development