Issue Details (XML | Word | Printable)

Key: HADOOP-2421
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Minor Minor
Assignee: Doug Cutting
Reporter: Nigel Daley
Votes: 1
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Hadoop Common

Release JDiff report of changes between different versions of Hadoop

Created: 13/Dec/07 11:12 PM   Updated: 04/Nov/08 06:48 PM
Return to search
Component/s: documentation
Affects Version/s: None
Fix Version/s: 0.18.2

Time Tracking:
Not Specified

File Attachments:
  Size
File Licensed for inclusion in ASF works 2421.patch.1 2008-01-21 07:28 AM Hrishikesh 3 kB
Text File Licensed for inclusion in ASF works HADOOP-2421.patch 2008-09-25 07:57 PM Doug Cutting 1.84 MB
Issue Links:
Reference
 

Resolution Date: 25/Sep/08 11:10 PM


 Description  « Hide
Similar to LUCENE-1083, it would be useful to report javadoc differences (ala JDiff) between Hadoop releases.

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Hrishikesh added a comment - 16/Jan/08 11:02 AM
Is there a URI where the old documentation is available ? or does the used need to download it and then pass the location using -D to the diff target ?

Doug Cutting added a comment - 16/Jan/08 06:53 PM
> Is there a URI where the old documentation is available ?

There are public URIs where released javadocs may be obtained, but I don't think JDiff uses the normal javadoc, but rather special javadoc output that it generates.

Please see LUCENE-1083 which addresses this further.


Hrishikesh added a comment - 17/Jan/08 06:19 AM
I had a look at the patch LUCENE-1083. Looks like Jdiff requires 2 sets of java code.

We have one set of java code (current / latest) in the svn tree, from where do i get the "OLD" one ?

e.g. if I need to run jdiff today i will have trunk checked out onto my machine (and the current version in trunk is 1.6 ). Now I want to Jdff the trunk and 1.3 .. where do i get 1.3 code ? Does the user need to download and then pass the path to ant with -D option ?


Doug Cutting added a comment - 17/Jan/08 07:13 PM
> where do i get the "OLD" one ?

The approach I suggested in the Lucene issue was to have some ant properties that determine the subversion tag url of the prior version. In trunk this would point to the prior release. We'd update it in trunk after each release is made. Then the ant build script would check this out in build/ if it didn't already exist there. We could (and should) permit this to be optimized, perhaps by permitting folks to override a property so that the prior version is stored somewhere more permanent than build/, and perhaps use 'svn switch; svn update' to make sure that the cached prior version contains what we expect.

> Does the user need to download and then pass the path to ant with -D option ?

I'd imagined that specifying -Djdiff.prior.dir would be optional, but would help performance a lot, but we could make it mandatory, and emit an error if it's not specified. That might reduce the load on subversion somewhat.


Hrishikesh added a comment - 18/Jan/08 06:41 AM

I'd imagined that specifying -Djdiff.prior.dir would be optional, but would help performance a lot, but we could make it mandatory, and emit an error if it's not specified. That might reduce the load on subversion somewhat.

Yes, I would like to make -Djdiff.prior.dir mandatory ..

Please comment


Hrishikesh added a comment - 21/Jan/08 07:28 AM
Try out the attached patch

The correct way to run the jdiff target is (You will need to chage the values as per your environment)

ant -Djdiff.prior.dir=/home/hadoop/myprojects/hadoop-0.13/src/java -Djdiff.prior.version=0.13 -DJDIFF_HOME=/home/hadoop/JDiff/jdiff-1.1.0/ jdiff


Doug Cutting added a comment - 25/Sep/08 05:20 PM
Should we target this for 0.18.2 and 0.19.0? HADOOP-3532 was committed in 0.18 along with the jdiff XML for 0.17. So we should include 0.17->0.18 API diffs in 0.18.2 and 0.18->-.19 API diffs in 0.19.0, no?

Doug Cutting added a comment - 25/Sep/08 07:57 PM
This better integrates jdiff into our release build process. After this is applied, if one adds the 'api-report' target to release builds, then jdiff output will be included in the documentation. Once this is committed I will update the HowToRelease page in the wiki.

Doug Cutting added a comment - 25/Sep/08 08:05 PM
To test this:
  • apply the patch
  • download Owen's modified jdiff from HADOOP-3532
  • ant -Djdiff.home=XXX api-report
  • ant -Djava5.home=YYY -Dforrest.home=ZZZ docs
  • ant package

Then browse build/hadoop-X.Y.Z/docs (the candidate release build) and look for the "API Changes" link.


Owen O'Malley added a comment - 25/Sep/08 10:21 PM
+1

Doug Cutting added a comment - 25/Sep/08 11:10 PM
I just committed this to both branches, and also updated the HowToRelease page in the wiki with instructions on how to generate the jdiff documentation.

Hadoop QA added a comment - 25/Sep/08 11:31 PM
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12390945/HADOOP-2421.patch
against trunk revision 699092.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 Eclipse classpath. The patch retains Eclipse classpath integrity.

+1 core tests. The patch passed core unit tests.

+1 contrib tests. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/console

This message is automatically generated.


Hudson added a comment - 26/Sep/08 05:03 PM

Hudson added a comment - 27/Sep/08 01:22 PM

Hudson added a comment - 29/Oct/08 02:39 PM
Integrated in Hadoop-trunk #646 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/646/)
Moving from 0.19 to 0.18.2 and marking it as a NEW FEATURE.