The HCatalog code needs to be moved into Hive from the incubator. This will be done by an 'svn mv' command in order to preserve history. I propose to do this as follows:

1. I will move all of the hcat code as is under the hcatalog stub directory created in HIVE-4145. I will put it into a sub-directory to avoid clashes with files names that have already been created (such as build.xml). The command for this will look like: 'svn mv https://svn.apache.org/repos/asf//incubator/hcatalog/trunk https://svn.apache.org/repos/asf//hive/trunk/hcatalog/hcat-code-dump' This is a call on the repo and cannot be uploaded as a patch to JIRA.
2. I can then deal with the files in that directory in separate commits.
   1. DISCLAIMER.txt can be removed as HCat is no longer incubating
   2. KEYS, LICENSE.txt, and NOTICE.TXT can be merged with the top level Hive files, as HCat no longer needs to track these entities separately.
   3. I propose we put CHANGES.txt, README.txt, and RELEASE_NOTES.txt in an newly created 'pre-hive-archive' directory. Going forward HCat will no longer need separate versions of these files. But for the sake of history it would be nice to keep them around.
   4. build.xml will be merged with the build.xml that was created in HIVE-4145
   5. The other directories and files will be moved to the top level.
   6. The Java code in these directories will be refactored into org.apache.hive.hcatalog and stub classes for org.apache.hcatalog classes will be created. This will only be done for classes available through public APIs.

I further propose that for the duration of these operations only, the hcatalog directory will be CTR (commit then review) so that I do not have to post patches and wait for a review for each of these file move/merge operations. I will have JIRAs for each of these operations so that they are trackable in the future.

I am proposing this double hop (move to hcat-code-dump and then from there) for two reasons. One, trying to do this in one patch is not doable based on how svn mv works. Two, it will be much easier to track and understand what is happening (both now and for users in the future reading the svn history) if it is done in a set of discrete steps rather than in one mega-move that also removes, moves, and merges files and directories.

+1 Plan looks reasonable to me.

Mithun pointed out that I'm only proposing how to move trunk here, not the branches, tags, and site sections. These will need to be moved as well. I don't think it makes any sense to merge these into Hive, Especially since we want to be able to release a 0.5.1 in the future if we need to, or if users wish to continue to build off of the existing branches without taking in the merge into Hive.

So, I modify my proposal as follows: I'll move all of the existing HCatalog code (trunk, branches, tags, site) into hive/hcatalog/hcatalog-historical. I'll then move/re-arrange ONLY the trunk section of the code, as outlined above. So when I'm done we'll have the following:

hive/
  hcatalog/
    build.xml
    core/
    hcatalog-pig-adapter/
    ...
    hcatalog-historical/
      branches/
      site/
      tags/

Taken from hive dev list:

What changes will be needed to make HCatalog's documentation set buildable
in its Hive location? Or will all HCatalog docs need to be converted to
wikidocs?

(Tech note: HCatalog has xdocs built with Forrest, unlike the dearly
departed Hive xdocs which were built with Anakia.)

– Lefty

This was covered in the original proposal for merging HCatalog into Hive:

5) Website
We will need to integrate HCatalog's website with Hive's. This should be
easy except for the documentation. HCat uses forrest for docs, Hive uses
wiki. We will need to put links under 'Documentation' for older versions
of HCat docs so users can find them. As far as how docs are handled for
the next version of HCatalog, I think that depends on the answer to
question 7 (next release of HCat), but I propose that HCat needs to conform
to the way Hive does docs on wiki. Though I would strongly encourage the
HCat docs to be version specific (that is, have a set of wiki pages for
each version). incubator.apache.org/hcatalog should be changed to
forward to hive.apache.org.

So in the initial checkin they will be built still with Forrest. We need to move them into the wiki to conform to the way Hive does docs.

(As a checkpoint, as of revision 651fd3acebd1e54a203824e9e8a0a8d6add21875, hcat trunk from inside hive compiles cleanly and all tests pass successfully)

Hey all - great work with getting the code into the Hive repo so far! As we make progress here I'd like to get everyone on the same page about what our final product is. That is, when someone builds a Hive package (ant clean package) how does HCatalog fit in? Do we continue to release two artifacts (separate Hive & HCatalog releases) or just one Hive artifact that also contains HCatalog? Once we're all on the same page here (I strongly prefer one Hive artifact that also contains HCatalog) we can figure out what the merged release looks like.

THoughts?

The plan of record is for HCatalog to be one of the artifacts contained in the Hive release.

We definitely want them to be one release with one artifact containing all of Hive (which now also includes the HCat jars and executables). The only case I could see for a separate HCat release is if there's a need to do a 0.5.1 release to address bugs in 0.5.0.