Hadoop Common
  1. Hadoop Common
  2. HADOOP-2421

Release JDiff report of changes between different versions of Hadoop

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.18.2
    • Component/s: documentation
    • Labels:
      None

      Description

      Similar to LUCENE-1083, it would be useful to report javadoc differences (ala JDiff) between Hadoop releases.

      1. 2421.patch.1
        3 kB
        Hrishikesh
      2. HADOOP-2421.patch
        1.84 MB
        Doug Cutting

        Issue Links

          Activity

          Hide
          Hrishikesh added a comment -

          Is there a URI where the old documentation is available ? or does the used need to download it and then pass the location using -D to the diff target ?

          Show
          Hrishikesh added a comment - Is there a URI where the old documentation is available ? or does the used need to download it and then pass the location using -D to the diff target ?
          Hide
          Doug Cutting added a comment -

          > Is there a URI where the old documentation is available ?

          There are public URIs where released javadocs may be obtained, but I don't think JDiff uses the normal javadoc, but rather special javadoc output that it generates.

          Please see LUCENE-1083 which addresses this further.

          Show
          Doug Cutting added a comment - > Is there a URI where the old documentation is available ? There are public URIs where released javadocs may be obtained, but I don't think JDiff uses the normal javadoc, but rather special javadoc output that it generates. Please see LUCENE-1083 which addresses this further.
          Hide
          Hrishikesh added a comment -

          I had a look at the patch LUCENE-1083. Looks like Jdiff requires 2 sets of java code.

          We have one set of java code (current / latest) in the svn tree, from where do i get the "OLD" one ?

          e.g. if I need to run jdiff today i will have trunk checked out onto my machine (and the current version in trunk is 1.6 ). Now I want to Jdff the trunk and 1.3 .. where do i get 1.3 code ? Does the user need to download and then pass the path to ant with -D option ?

          Show
          Hrishikesh added a comment - I had a look at the patch LUCENE-1083 . Looks like Jdiff requires 2 sets of java code. We have one set of java code (current / latest) in the svn tree, from where do i get the "OLD" one ? e.g. if I need to run jdiff today i will have trunk checked out onto my machine (and the current version in trunk is 1.6 ). Now I want to Jdff the trunk and 1.3 .. where do i get 1.3 code ? Does the user need to download and then pass the path to ant with -D option ?
          Hide
          Doug Cutting added a comment -

          > where do i get the "OLD" one ?

          The approach I suggested in the Lucene issue was to have some ant properties that determine the subversion tag url of the prior version. In trunk this would point to the prior release. We'd update it in trunk after each release is made. Then the ant build script would check this out in build/ if it didn't already exist there. We could (and should) permit this to be optimized, perhaps by permitting folks to override a property so that the prior version is stored somewhere more permanent than build/, and perhaps use 'svn switch; svn update' to make sure that the cached prior version contains what we expect.

          > Does the user need to download and then pass the path to ant with -D option ?

          I'd imagined that specifying -Djdiff.prior.dir would be optional, but would help performance a lot, but we could make it mandatory, and emit an error if it's not specified. That might reduce the load on subversion somewhat.

          Show
          Doug Cutting added a comment - > where do i get the "OLD" one ? The approach I suggested in the Lucene issue was to have some ant properties that determine the subversion tag url of the prior version. In trunk this would point to the prior release. We'd update it in trunk after each release is made. Then the ant build script would check this out in build/ if it didn't already exist there. We could (and should) permit this to be optimized, perhaps by permitting folks to override a property so that the prior version is stored somewhere more permanent than build/, and perhaps use 'svn switch; svn update' to make sure that the cached prior version contains what we expect. > Does the user need to download and then pass the path to ant with -D option ? I'd imagined that specifying -Djdiff.prior.dir would be optional, but would help performance a lot, but we could make it mandatory, and emit an error if it's not specified. That might reduce the load on subversion somewhat.
          Hide
          Hrishikesh added a comment -

          I'd imagined that specifying -Djdiff.prior.dir would be optional, but would help performance a lot, but we could make it mandatory, and emit an error if it's not specified. That might reduce the load on subversion somewhat.

          Yes, I would like to make -Djdiff.prior.dir mandatory ..

          Please comment

          Show
          Hrishikesh added a comment - I'd imagined that specifying -Djdiff.prior.dir would be optional, but would help performance a lot, but we could make it mandatory, and emit an error if it's not specified. That might reduce the load on subversion somewhat. Yes, I would like to make -Djdiff.prior.dir mandatory .. Please comment
          Hide
          Hrishikesh added a comment -

          Try out the attached patch

          The correct way to run the jdiff target is (You will need to chage the values as per your environment)

          ant -Djdiff.prior.dir=/home/hadoop/myprojects/hadoop-0.13/src/java -Djdiff.prior.version=0.13 -DJDIFF_HOME=/home/hadoop/JDiff/jdiff-1.1.0/ jdiff

          Show
          Hrishikesh added a comment - Try out the attached patch The correct way to run the jdiff target is (You will need to chage the values as per your environment) ant -Djdiff.prior.dir=/home/hadoop/myprojects/hadoop-0.13/src/java -Djdiff.prior.version=0.13 -DJDIFF_HOME=/home/hadoop/JDiff/jdiff-1.1.0/ jdiff
          Hide
          Doug Cutting added a comment -

          Should we target this for 0.18.2 and 0.19.0? HADOOP-3532 was committed in 0.18 along with the jdiff XML for 0.17. So we should include 0.17->0.18 API diffs in 0.18.2 and 0.18->-.19 API diffs in 0.19.0, no?

          Show
          Doug Cutting added a comment - Should we target this for 0.18.2 and 0.19.0? HADOOP-3532 was committed in 0.18 along with the jdiff XML for 0.17. So we should include 0.17->0.18 API diffs in 0.18.2 and 0.18->-.19 API diffs in 0.19.0, no?
          Hide
          Doug Cutting added a comment -

          This better integrates jdiff into our release build process. After this is applied, if one adds the 'api-report' target to release builds, then jdiff output will be included in the documentation. Once this is committed I will update the HowToRelease page in the wiki.

          Show
          Doug Cutting added a comment - This better integrates jdiff into our release build process. After this is applied, if one adds the 'api-report' target to release builds, then jdiff output will be included in the documentation. Once this is committed I will update the HowToRelease page in the wiki.
          Hide
          Doug Cutting added a comment -

          To test this:

          • apply the patch
          • download Owen's modified jdiff from HADOOP-3532
          • ant -Djdiff.home=XXX api-report
          • ant -Djava5.home=YYY -Dforrest.home=ZZZ docs
          • ant package

          Then browse build/hadoop-X.Y.Z/docs (the candidate release build) and look for the "API Changes" link.

          Show
          Doug Cutting added a comment - To test this: apply the patch download Owen's modified jdiff from HADOOP-3532 ant -Djdiff.home=XXX api-report ant -Djava5.home=YYY -Dforrest.home=ZZZ docs ant package Then browse build/hadoop-X.Y.Z/docs (the candidate release build) and look for the "API Changes" link.
          Hide
          Owen O'Malley added a comment -

          +1

          Show
          Owen O'Malley added a comment - +1
          Hide
          Doug Cutting added a comment -

          I just committed this to both branches, and also updated the HowToRelease page in the wiki with instructions on how to generate the jdiff documentation.

          Show
          Doug Cutting added a comment - I just committed this to both branches, and also updated the HowToRelease page in the wiki with instructions on how to generate the jdiff documentation.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12390945/HADOOP-2421.patch
          against trunk revision 699092.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12390945/HADOOP-2421.patch against trunk revision 699092. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3372/console This message is automatically generated.
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #615 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/615/ )
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #616 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/616/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk #646 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/646/)
          Moving from 0.19 to 0.18.2 and marking it as a NEW FEATURE.

          Show
          Hudson added a comment - Integrated in Hadoop-trunk #646 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/646/ ) Moving from 0.19 to 0.18.2 and marking it as a NEW FEATURE.

            People

            • Assignee:
              Doug Cutting
              Reporter:
              Nigel Daley
            • Votes:
              1 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development