Lucene - Core
  1. Lucene - Core
  2. LUCENE-3228

build should allow you (especially hudson) to refer to a local javadocs installation instead of downloading

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None

      Description

      Currently, we fail on all javadocs warnings.

      However, you get a warning if it cannot download the package-list from sun.com
      So I think we should allow you optionally set a sysprop using linkoffline.
      Then we would get much less hudson "fake failures"

      I feel like Mike opened an issue for this already but I cannot find it.

      1. LUCENE-3228.trunk.patch
        15 kB
        Steve Rowe
      2. LUCENE-3228.branch_3x.patch
        16 kB
        Steve Rowe
      3. LUCENE-3228.patch
        13 kB
        Hoss Man

        Issue Links

          Activity

          Robert Muir created issue -
          Robert Muir made changes -
          Field Original Value New Value
          Assignee Robert Muir [ rcmuir ]
          Hide
          Robert Muir added a comment -

          as a start, i installed the two freebsd ports for java doc on hudson into /usr/local/share/doc/jdk1.5 and jdk1.6

          I'll see if i can add the hooks to the build scripts now

          Show
          Robert Muir added a comment - as a start, i installed the two freebsd ports for java doc on hudson into /usr/local/share/doc/jdk1.5 and jdk1.6 I'll see if i can add the hooks to the build scripts now
          rmuir committed 1138418 (3 files)
          Reviews: none

          LUCENE-3228: use an offline javadoc link for the 30 minute tests-only builds

          Hide
          Robert Muir added a comment -

          As a partial solution, I setup the 30 minute builds to just directly override javadoc.link (and javadoc.link.java for Solr) for our 30 minute builds... we don't care about the actual javadoc artifacts or where the links actually point to, only that there are no warnings.

          This is in r1138418

          Show
          Robert Muir added a comment - As a partial solution, I setup the 30 minute builds to just directly override javadoc.link (and javadoc.link.java for Solr) for our 30 minute builds... we don't care about the actual javadoc artifacts or where the links actually point to, only that there are no warnings. This is in r1138418
          Hide
          Robert Muir added a comment -

          I noticed also that solr uses an online link for junit javadocs... we should download this one and do the same, too.
          I'll look at this if the link for the sun javadocs "takes" for the 30 minute builds.

          Show
          Robert Muir added a comment - I noticed also that solr uses an online link for junit javadocs... we should download this one and do the same, too. I'll look at this if the link for the sun javadocs "takes" for the 30 minute builds.
          Hide
          Hoss Man added a comment -

          +1

          I think we should allow you optionally set a sysprop using linkoffline

          hell, why bother with the sysprop? .. lets just commit the package-list files for all third party libs we use into dev-tools and completely eliminate the need for net when building javadocs.

          Show
          Hoss Man added a comment - +1 I think we should allow you optionally set a sysprop using linkoffline hell, why bother with the sysprop? .. lets just commit the package-list files for all third party libs we use into dev-tools and completely eliminate the need for net when building javadocs.
          Hide
          Michael McCandless added a comment -

          lets just commit the package-list files for all third party libs we use into dev-tools and completely eliminate the need for net when building javadocs.

          +1

          Hitting build failures because we can't download these package lists is silly.

          Show
          Michael McCandless added a comment - lets just commit the package-list files for all third party libs we use into dev-tools and completely eliminate the need for net when building javadocs. +1 Hitting build failures because we can't download these package lists is silly.
          Hide
          Robert Muir added a comment -

          I agree with hossman too. I'm just a javadocs dummy and was doing what I could to stop the 30minute builds.

          I cant figure out this linkoffline (at least with my experiments its confusing)... but this sounds great.

          Show
          Robert Muir added a comment - I agree with hossman too. I'm just a javadocs dummy and was doing what I could to stop the 30minute builds. I cant figure out this linkoffline (at least with my experiments its confusing)... but this sounds great.
          Hide
          Hoss Man added a comment -

          rmuir: here's a rough patch showing how the link offline stuff works. (as far as i understand it anyway)

          some quick testing didn't turn up any problems, but i didn't test the where modules/contribs usage of invoke-javadoc.

          there may be cleanup we want to do to - for now i avoided adding more sys properties for the package-list dirs, but maybe we want them? i dunno. there 's also some existing instances of the "<link>" tag that look totally bogus and broken (see the WTF comments i added) but i didn't test what changes if i remove them

          this patch should also fix SOLR-2439 (use relative links for lucene jdocs from solr jdocs.

          Show
          Hoss Man added a comment - rmuir: here's a rough patch showing how the link offline stuff works. (as far as i understand it anyway) some quick testing didn't turn up any problems, but i didn't test the where modules/contribs usage of invoke-javadoc. there may be cleanup we want to do to - for now i avoided adding more sys properties for the package-list dirs, but maybe we want them? i dunno. there 's also some existing instances of the "<link>" tag that look totally bogus and broken (see the WTF comments i added) but i didn't test what changes if i remove them this patch should also fix SOLR-2439 (use relative links for lucene jdocs from solr jdocs.
          Hoss Man made changes -
          Attachment LUCENE-3228.patch [ 12484005 ]
          Hide
          Robert Muir added a comment -

          I am glad you had the same "WTF", although ant docs say its ok to use both, the current tasks in e.g. lucene have both the link attribute and nested link-without-href-wtf, and as i tried mixing linkoffline in different ways, it would appear to work, until i changed the link to javaBROKENURL.sun.com/...., etc.

          I think we should go with this patch so we aren't downloading this junk anymore, it causes false build failures, the only trick I can think of is how to ensure lucene source releases build by themself without reaching back to dev-tools (i think this is broken on trunk at the moment, but it does work on 3.x right now)

          Show
          Robert Muir added a comment - I am glad you had the same "WTF", although ant docs say its ok to use both, the current tasks in e.g. lucene have both the link attribute and nested link-without-href-wtf, and as i tried mixing linkoffline in different ways, it would appear to work, until i changed the link to javaBROKENURL.sun.com/...., etc. I think we should go with this patch so we aren't downloading this junk anymore, it causes false build failures, the only trick I can think of is how to ensure lucene source releases build by themself without reaching back to dev-tools (i think this is broken on trunk at the moment, but it does work on 3.x right now)
          Hide
          Steve Rowe added a comment -

          I propose switching to the oracle.com link suggested by Chris Male:

          http://download.oracle.com/javase/6/docs/api/package-list apparently works reliably.

          This would be lots simpler than trying to figure out dev-tools etc., assuming that this link is indeed reliable.

          Show
          Steve Rowe added a comment - I propose switching to the oracle.com link suggested by Chris Male : http://download.oracle.com/javase/6/docs/api/package-list apparently works reliably. This would be lots simpler than trying to figure out dev-tools etc., assuming that this link is indeed reliable.
          Hide
          Chris Male added a comment -

          +1

          Show
          Chris Male added a comment - +1
          Hide
          Michael McCandless added a comment -

          +1 let's try this and see if it is indeed reliable.

          Show
          Michael McCandless added a comment - +1 let's try this and see if it is indeed reliable.
          Hide
          Uwe Schindler added a comment -

          Just one other idea:

          • We already have JAVA_HOME set (direct or implicitely set by ANT)
          • The Javadocs are always at same location in $JAVA_HOME

          Could we not use this to point to the package list (at least fpr the JDK part). I don't like the hardcoded package list.

          Show
          Uwe Schindler added a comment - Just one other idea: We already have JAVA_HOME set (direct or implicitely set by ANT) The Javadocs are always at same location in $JAVA_HOME Could we not use this to point to the package list (at least fpr the JDK part). I don't like the hardcoded package list.
          Hide
          Steve Rowe added a comment -

          The Javadocs are always at same location in $JAVA_HOME

          I looked at JAVA_HOME on Windows for 1.5.0_22 and 1.6.0_23 (both 64 bit JDKs), and neither included Javadocs. Maybe they're separately downloadable?

          Show
          Steve Rowe added a comment - The Javadocs are always at same location in $JAVA_HOME I looked at JAVA_HOME on Windows for 1.5.0_22 and 1.6.0_23 (both 64 bit JDKs), and neither included Javadocs. Maybe they're separately downloadable?
          Hide
          Uwe Schindler added a comment -

          Sorry, you are right. I have it here, but the README.html in JDK's root folder says:

          JDK(TM) Documentation

          The on-line JavaTM Platform, Standard Edition (Java SE) Documentation contains API specifications, feature descriptions, developer guides, reference pages for JDKTM tools and utilities, demos, and links to related information. This documentation is also available in a download bundle which you can install on your machine. To obtain the documentation bundle, see the download page. For API documentation, refer to the The JavaTM Platform, Standard Edition API Specification This provides brief descriptions of the API with an emphasis on specifications, not on code examples.

          Show
          Uwe Schindler added a comment - Sorry, you are right. I have it here, but the README.html in JDK's root folder says: JDK(TM) Documentation The on-line JavaTM Platform, Standard Edition (Java SE) Documentation contains API specifications, feature descriptions, developer guides, reference pages for JDKTM tools and utilities, demos, and links to related information. This documentation is also available in a download bundle which you can install on your machine. To obtain the documentation bundle, see the download page. For API documentation, refer to the The JavaTM Platform, Standard Edition API Specification This provides brief descriptions of the API with an emphasis on specifications, not on code examples.
          sarowe committed 1143133 (2 files)
          Reviews: none

          LUCENE-3228: Switch online javadoc links to a hopefully more reliable URL.

          sarowe committed 1143142 (2 files)
          Reviews: none

          LUCENE-3228: Switch online javadoc links to a hopefully more reliable URL.

          Hoss Man made changes -
          Link This issue relates to LUCENE-3587 [ LUCENE-3587 ]
          Hide
          Hoss Man added a comment -

          sarowe asked about this issue in LUCENE-3587.

          FWIW, i thought this issue (LUCENE-3228) had been resolved a long time ago based on some comments i remember people making, but evidently those comments where on irc/mail and folks didn't post them in Jira.

          Problems with the patch i attached (that i know of):

          1) we don't distribute dev-tools in our releases, so at a minimum we'd need to find a new home for any package-list files we wanted to ship.

          2) the Java documentation from Oracle has some licensing/restrictions that affect redistribution which don't seem to be compatible with ASF 3rd party licensing policy so we can't include the java package-list files in our releases

          ...we could still use the ideas in this patch to deal with package-list files for non java/oracle distrobutions, but at the time this patch was written the only other extneral javadocs we linked to where that might be useful was junit, and since this patch was created, that link has just been removed outright from our build.xml files.

          I don't think there's anything left here but to resolve as Won't Fix

          Show
          Hoss Man added a comment - sarowe asked about this issue in LUCENE-3587 . FWIW, i thought this issue ( LUCENE-3228 ) had been resolved a long time ago based on some comments i remember people making, but evidently those comments where on irc/mail and folks didn't post them in Jira. Problems with the patch i attached (that i know of): 1) we don't distribute dev-tools in our releases, so at a minimum we'd need to find a new home for any package-list files we wanted to ship. 2) the Java documentation from Oracle has some licensing/restrictions that affect redistribution which don't seem to be compatible with ASF 3rd party licensing policy so we can't include the java package-list files in our releases ...we could still use the ideas in this patch to deal with package-list files for non java/oracle distrobutions, but at the time this patch was written the only other extneral javadocs we linked to where that might be useful was junit, and since this patch was created, that link has just been removed outright from our build.xml files. I don't think there's anything left here but to resolve as Won't Fix
          Hide
          Steve Rowe added a comment -

          I don't think there's anything left here but to resolve as Won't Fix

          I disagree.

          1) we don't distribute dev-tools in our releases, so at a minimum we'd need to find a new home for any package-list files we wanted to ship.

          How does lucene/src/tools/javadoc/ grab you?

          2) the Java documentation from Oracle has some licensing/restrictions that affect redistribution which don't seem to be compatible with ASF 3rd party licensing policy so we can't include the java package-list files in our releases

          Here is a direct link to the Sun/Oracle documentation redistribution restrictions: http://java.sun.com/docs/redist.html.

          We can include the Java javadoc package-list file in Subversion but exclude it from our source releases, and include a mechanism to download it when it's absent (i.e., in the source release).

          There is an ASF precedent for redistributing Java javadoc package-list files in the Maven project's javadoc plugin: http://jira.codehaus.org/browse/MJAVADOC-315 and https://jira.codehaus.org/browse/MJAVADOC-327 - in Subversion at http://svn.apache.org/viewvc/maven/plugins/trunk/maven-javadoc-plugin/src/main/resources/org/apache/maven/plugin/javadoc/. I can't find any associated discussion of legal ramifications, though.

          I'll put up a patch shortly implementing these ideas.

          Show
          Steve Rowe added a comment - I don't think there's anything left here but to resolve as Won't Fix I disagree. 1) we don't distribute dev-tools in our releases, so at a minimum we'd need to find a new home for any package-list files we wanted to ship. How does lucene/src/tools/javadoc/ grab you? 2) the Java documentation from Oracle has some licensing/restrictions that affect redistribution which don't seem to be compatible with ASF 3rd party licensing policy so we can't include the java package-list files in our releases Here is a direct link to the Sun/Oracle documentation redistribution restrictions: http://java.sun.com/docs/redist.html . We can include the Java javadoc package-list file in Subversion but exclude it from our source releases, and include a mechanism to download it when it's absent (i.e., in the source release). There is an ASF precedent for redistributing Java javadoc package-list files in the Maven project's javadoc plugin: http://jira.codehaus.org/browse/MJAVADOC-315 and https://jira.codehaus.org/browse/MJAVADOC-327 - in Subversion at http://svn.apache.org/viewvc/maven/plugins/trunk/maven-javadoc-plugin/src/main/resources/org/apache/maven/plugin/javadoc/ . I can't find any associated discussion of legal ramifications, though. I'll put up a patch shortly implementing these ideas.
          Hide
          Steve Rowe added a comment -

          Patch for branch_3x. Features:

          1. Adds package-list files for Oracle Java javadocs and JUnit javadocs to subversion.
          2. When building Lucene and Solr source releases, the Oracle Java javadocs package-list file is removed.
          3. When connected or disconnected from the network, javadocs built from a subversion checkout contain links to Oracle javadocs.
          4. When connected to the network, javadocs built from a source release will attempt to download the Oracle Java package-list file.
          5. When the Oracle Java package-list file is not present, either because the user is building from a source release while disconnected from the network, or because the package-list file for Oracle Java javadocs is not downloadable for some other reason, javadocs will be built and the build will not fail, though an error will appear in the build log.
          6. Links from Solr javadocs to Lucene's javadocs are enabled. When building a non-release version, the links are to the most recently built nightly Jenkins javadocs, as in Hoss's patch on this issue. When building a release version, links are to the same-versioned Lucene release javadocs.
          Show
          Steve Rowe added a comment - Patch for branch_3x. Features: Adds package-list files for Oracle Java javadocs and JUnit javadocs to subversion. When building Lucene and Solr source releases, the Oracle Java javadocs package-list file is removed. When connected or disconnected from the network, javadocs built from a subversion checkout contain links to Oracle javadocs. When connected to the network, javadocs built from a source release will attempt to download the Oracle Java package-list file. When the Oracle Java package-list file is not present, either because the user is building from a source release while disconnected from the network, or because the package-list file for Oracle Java javadocs is not downloadable for some other reason, javadocs will be built and the build will not fail, though an error will appear in the build log. Links from Solr javadocs to Lucene's javadocs are enabled. When building a non-release version, the links are to the most recently built nightly Jenkins javadocs, as in Hoss's patch on this issue. When building a release version, links are to the same-versioned Lucene release javadocs.
          Steve Rowe made changes -
          Attachment LUCENE-3228.branch_3x.patch [ 12505181 ]
          Hide
          Steve Rowe added a comment -

          Trunk for patch with the same changes.

          Show
          Steve Rowe added a comment - Trunk for patch with the same changes.
          Steve Rowe made changes -
          Attachment LUCENE-3228.trunk.patch [ 12505182 ]
          Hide
          Steve Rowe added a comment -

          If there are no objections I will commit this tomorrow.

          Show
          Steve Rowe added a comment - If there are no objections I will commit this tomorrow.
          Steve Rowe made changes -
          Assignee Robert Muir [ rcmuir ] Steven Rowe [ steve_rowe ]
          sarowe committed 1210021 (1 file)
          Reviews: none

          LUCENE-3228: reverted accidentally committed debug printing

          Hide
          Steve Rowe added a comment -

          Committed:

          • r1210020: trunk
          • r1210022: branch_3x
          Show
          Steve Rowe added a comment - Committed: r1210020: trunk r1210022: branch_3x
          Steve Rowe made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Lucene Fields New [ 10121 ]
          Fix Version/s 3.6 [ 12319070 ]
          Fix Version/s 4.0 [ 12314025 ]
          Resolution Fixed [ 1 ]
          Uwe Schindler made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Steve Rowe
              Reporter:
              Robert Muir
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development