Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7651

Javadocs build fails with Java 8 update 121

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 6.4
    • Fix Version/s: 6.x, 6.5, 6.4.1, 7.0
    • Component/s: general/javadocs
    • Labels:
    • Environment:

      Java 8 update 121

    • Lucene Fields:
      New

      Description

      Oracle released the recent Java 8 security update (u121). The Jenkins builds fail with the following error while building the Javadocs:

        [javadoc] Constructing Javadoc information...
        [javadoc] javadoc: error - Argument for -bottom contains JavaScript.
        [javadoc] Use --allow-script-in-comments to allow use of JavaScript.
        [javadoc] 1 error
      

      This is caused by the Javascript added to pretty-print code examples. We load this in the page footer "<bottom>" parameter.

      Surely, it will be posisble to simply add the mentioned argument, but this will break builds with earlier Java 8 versions.

      This is nowhere documented, I haven't seen any documentation about this flag nowhere, so I assume this is a bug in Java. They can't change or add command line parameters in minor updates of Java 8. I will ask on the OpenJDK mailing lists if this is a bug (maybe accidentally backported from Java 9).

      1. LUCENE-7651.patch
        54 kB
        Uwe Schindler
      2. LUCENE-7651.patch
        51 kB
        Uwe Schindler
      3. LUCENE-7651.patch
        51 kB
        Uwe Schindler
      4. LUCENE-7651.patch
        3 kB
        Uwe Schindler

        Activity

        Hide
        thetaphi Uwe Schindler added a comment - - edited

        I set to "critical", because this also breaks our latest release 6.4, which can't be build from source with Java 8 update 121.

        Show
        thetaphi Uwe Schindler added a comment - - edited I set to "critical", because this also breaks our latest release 6.4, which can't be build from source with Java 8 update 121.
        Hide
        thetaphi Uwe Schindler added a comment -

        I will try to figure out if moving the Javascript to a separate file helps here. In HTML 5 inline Javascript in HTML files is a no-go.

        Show
        thetaphi Uwe Schindler added a comment - I will try to figure out if moving the Javascript to a separate file helps here. In HTML 5 inline Javascript in HTML files is a no-go.
        Hide
        thetaphi Uwe Schindler added a comment -

        I checked it out, it disallows any <script>, also externally linked ones in the -bottom parameter of Javadocs. This is amajor pain. As adding this special allow-javascript parameter is brekaing builds with older versions and its hard to detect the exact build version in Ant, we have 2 options:

        • Remove prettify.js
        • Inject the prettify code somehow else into the docs (e.g. by copying all script files together and append them to the javadoc-generated JS file after javadoc succeeds).
        Show
        thetaphi Uwe Schindler added a comment - I checked it out, it disallows any <script> , also externally linked ones in the -bottom parameter of Javadocs. This is amajor pain. As adding this special allow-javascript parameter is brekaing builds with older versions and its hard to detect the exact build version in Ant, we have 2 options: Remove prettify.js Inject the prettify code somehow else into the docs (e.g. by copying all script files together and append them to the javadoc-generated JS file after javadoc succeeds).
        Hide
        thetaphi Uwe Schindler added a comment -

        This implements first option:

         lucene/common-build.xml | 14 --------------
         1 file changed, 14 deletions(-)
        
        diff --git a/lucene/common-build.xml b/lucene/common-build.xml
        index 48cf457..61948bb 100644
        --- a/lucene/common-build.xml
        +++ b/lucene/common-build.xml
        @@ -2107,20 +2107,6 @@ ${ant.project.name}.test.dependencies=${test.classpath.list}
                 <link offline="true" href="${javadoc.link}" packagelistLoc="${javadoc.packagelist.dir}/java8"/>
                 <bottom><![CDATA[
                   <i>Copyright &copy; ${year} Apache Software Foundation.  All Rights Reserved.</i>
        -          <script src='{@docRoot}/prettify.js' type='text/javascript'></script>
        -          <script type='text/javascript'>
        -            (function(){
        -              var oldonload = window.onload;
        -              if (typeof oldonload != 'function') {
        -                window.onload = prettyPrint;
        -              } else {
        -                window.onload = function() {
        -                  oldonload();
        -                  prettyPrint();
        -                }
        -              }
        -            })();
        -          </script>
                 ]]></bottom>
                 
                 <sources />
        

        I checked our sources. We allready append the CSS of prettify to Javadocs's main CSS files. As Java 8 contains also a Javadocs-generated scripts.js file, we could do the same here. I am working on that...

        Show
        thetaphi Uwe Schindler added a comment - This implements first option: lucene/common-build.xml | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/lucene/common-build.xml b/lucene/common-build.xml index 48cf457..61948bb 100644 --- a/lucene/common-build.xml +++ b/lucene/common-build.xml @@ -2107,20 +2107,6 @@ ${ant.project.name}.test.dependencies=${test.classpath.list} <link offline="true" href="${javadoc.link}" packagelistLoc="${javadoc.packagelist.dir}/java8"/> <bottom><![CDATA[ <i>Copyright &copy; ${year} Apache Software Foundation. All Rights Reserved.</i> - <script src='{@docRoot}/prettify.js' type='text/javascript'></script> - <script type='text/javascript'> - (function(){ - var oldonload = window.onload; - if (typeof oldonload != 'function') { - window.onload = prettyPrint; - } else { - window.onload = function() { - oldonload(); - prettyPrint(); - } - } - })(); - </script> ]]></bottom> <sources /> I checked our sources. We allready append the CSS of prettify to Javadocs's main CSS files. As Java 8 contains also a Javadocs-generated scripts.js file, we could do the same here. I am working on that...
        Hide
        thetaphi Uwe Schindler added a comment - - edited

        This would be my proposal.

        Show
        thetaphi Uwe Schindler added a comment - - edited This would be my proposal.
        Hide
        thetaphi Uwe Schindler added a comment -

        Sorry last patch had a missing pair of brackets. This fixes it any also works locally. Stupid Javascript! KRRRR!

        Show
        thetaphi Uwe Schindler added a comment - Sorry last patch had a missing pair of brackets. This fixes it any also works locally. Stupid Javascript! KRRRR!
        Hide
        thetaphi Uwe Schindler added a comment -

        Attached is a patch that also updates Prettify to latest version. The used one produced Javascript errors in Chrome, so I updated. I also removed the useless additional language plugin files.

        Show
        thetaphi Uwe Schindler added a comment - Attached is a patch that also updates Prettify to latest version. The used one produced Javascript errors in Chrome, so I updated. I also removed the useless additional language plugin files.
        Hide
        thetaphi Uwe Schindler added a comment -

        Final updates, I think this is committable.

        Any suggestions?

        Show
        thetaphi Uwe Schindler added a comment - Final updates, I think this is committable. Any suggestions?
        Hide
        thetaphi Uwe Schindler added a comment -

        Add correct license header to CSS file.

        Show
        thetaphi Uwe Schindler added a comment - Add correct license header to CSS file.
        Hide
        thetaphi Uwe Schindler added a comment -

        I just checked. The "hack" also works with IBM J9, so I see no problem doing this. I was afraid that maybe IBM does not have script.js in the Javadoc output.

        Nevertheless, this is not risky like the previous hack. If the Javadocs generator no longer produces script.js or stylesheet.css the Code Prettify would just no longer work, but not break Javadocs or build.

        Show
        thetaphi Uwe Schindler added a comment - I just checked. The "hack" also works with IBM J9, so I see no problem doing this. I was afraid that maybe IBM does not have script.js in the Javadoc output. Nevertheless, this is not risky like the previous hack. If the Javadocs generator no longer produces script.js or stylesheet.css the Code Prettify would just no longer work, but not break Javadocs or build.
        Hide
        thetaphi Uwe Schindler added a comment -
        Show
        thetaphi Uwe Schindler added a comment - I sent the following message to OpenJDK: http://mail.openjdk.java.net/pipermail/javadoc-dev/2017-January/000281.html
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit ee5a36011220bd2a7a8e45de27d5321cc7610bff in lucene-solr's branch refs/heads/master from Uwe Schindler
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ee5a360 ]

        LUCENE-7651: Fix Javadocs build for Java 8u121 by injecting "Google Code Prettify" without adding Javascript to Javadocs's -bottom parameter. Also update Prettify to latest version to fix Google Chrome issue.

        Show
        jira-bot ASF subversion and git services added a comment - Commit ee5a36011220bd2a7a8e45de27d5321cc7610bff in lucene-solr's branch refs/heads/master from Uwe Schindler [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ee5a360 ] LUCENE-7651 : Fix Javadocs build for Java 8u121 by injecting "Google Code Prettify" without adding Javascript to Javadocs's -bottom parameter. Also update Prettify to latest version to fix Google Chrome issue.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit b808ee6099d03d1fdbccd5a17720069d74d0b8a4 in lucene-solr's branch refs/heads/branch_6x from Uwe Schindler
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b808ee6 ]

        LUCENE-7651: Fix Javadocs build for Java 8u121 by injecting "Google Code Prettify" without adding Javascript to Javadocs's -bottom parameter. Also update Prettify to latest version to fix Google Chrome issue.

        Show
        jira-bot ASF subversion and git services added a comment - Commit b808ee6099d03d1fdbccd5a17720069d74d0b8a4 in lucene-solr's branch refs/heads/branch_6x from Uwe Schindler [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b808ee6 ] LUCENE-7651 : Fix Javadocs build for Java 8u121 by injecting "Google Code Prettify" without adding Javascript to Javadocs's -bottom parameter. Also update Prettify to latest version to fix Google Chrome issue.
        Hide
        thetaphi Uwe Schindler added a comment - - edited

        I committed this to master and branch_6x. I will now disable all 6.4 branch builds. If we will have a further bugfix release on this branch, we should backport this. Please reopen in that case.

        Show
        thetaphi Uwe Schindler added a comment - - edited I committed this to master and branch_6x. I will now disable all 6.4 branch builds. If we will have a further bugfix release on this branch, we should backport this. Please reopen in that case.
        Hide
        thetaphi Uwe Schindler added a comment -

        Fixed for now, please reopen for every bugfix release on any branch that was not yet updated to use this.

        Show
        thetaphi Uwe Schindler added a comment - Fixed for now, please reopen for every bugfix release on any branch that was not yet updated to use this.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit ee5a36011220bd2a7a8e45de27d5321cc7610bff in lucene-solr's branch refs/heads/apiv2 from Uwe Schindler
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ee5a360 ]

        LUCENE-7651: Fix Javadocs build for Java 8u121 by injecting "Google Code Prettify" without adding Javascript to Javadocs's -bottom parameter. Also update Prettify to latest version to fix Google Chrome issue.

        Show
        jira-bot ASF subversion and git services added a comment - Commit ee5a36011220bd2a7a8e45de27d5321cc7610bff in lucene-solr's branch refs/heads/apiv2 from Uwe Schindler [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ee5a360 ] LUCENE-7651 : Fix Javadocs build for Java 8u121 by injecting "Google Code Prettify" without adding Javascript to Javadocs's -bottom parameter. Also update Prettify to latest version to fix Google Chrome issue.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 680153de29c5b01d4a8afad88d4a7b84ab01e145 in lucene-solr's branch refs/heads/branch_6_4 from Uwe Schindler
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=680153d ]

        LUCENE-7651: Fix Javadocs build for Java 8u121 by injecting "Google Code Prettify" without adding Javascript to Javadocs's -bottom parameter. Also update Prettify to latest version to fix Google Chrome issue.

        1. Conflicts:
        2. lucene/CHANGES.txt
        Show
        jira-bot ASF subversion and git services added a comment - Commit 680153de29c5b01d4a8afad88d4a7b84ab01e145 in lucene-solr's branch refs/heads/branch_6_4 from Uwe Schindler [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=680153d ] LUCENE-7651 : Fix Javadocs build for Java 8u121 by injecting "Google Code Prettify" without adding Javascript to Javadocs's -bottom parameter. Also update Prettify to latest version to fix Google Chrome issue. Conflicts: lucene/CHANGES.txt
        Hide
        thetaphi Uwe Schindler added a comment -

        BTW, the release notes of Java 8u121 was updated to mention this change: http://www.oracle.com/technetwork/java/javase/8u121-relnotes-3315208.html
        It still breaks our previous releases, but we can tell people that its caused by Oracle, not us.

        Show
        thetaphi Uwe Schindler added a comment - BTW, the release notes of Java 8u121 was updated to mention this change: http://www.oracle.com/technetwork/java/javase/8u121-relnotes-3315208.html It still breaks our previous releases, but we can tell people that its caused by Oracle, not us.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit d3a2ed8487f3934d53f864b092f91966814d4cf7 in lucene-solr's branch refs/heads/branch_6x from Adrien Grand
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d3a2ed8 ]

        LUCENE-7651: Move under the 6.4.1 section.

        Show
        jira-bot ASF subversion and git services added a comment - Commit d3a2ed8487f3934d53f864b092f91966814d4cf7 in lucene-solr's branch refs/heads/branch_6x from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d3a2ed8 ] LUCENE-7651 : Move under the 6.4.1 section.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 9dcfcb6e6fcbab36d50c4baca697c1104c2a72ff in lucene-solr's branch refs/heads/master from Adrien Grand
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9dcfcb6 ]

        LUCENE-7651: Move under the 6.4.1 section.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 9dcfcb6e6fcbab36d50c4baca697c1104c2a72ff in lucene-solr's branch refs/heads/master from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9dcfcb6 ] LUCENE-7651 : Move under the 6.4.1 section.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit cffa82062c5be766db6bc87bc232a39a413600ec in lucene-solr's branch refs/heads/branch_5_5 from Uwe Schindler
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cffa820 ]

        LUCENE-7651: Fix Javadocs build for Java 8u121 by injecting "Google Code Prettify" without adding Javascript to Javadocs's -bottom parameter. Also update Prettify to latest version to fix Google Chrome issue.

        1. Conflicts:
        2. lucene/CHANGES.txt
        Show
        jira-bot ASF subversion and git services added a comment - Commit cffa82062c5be766db6bc87bc232a39a413600ec in lucene-solr's branch refs/heads/branch_5_5 from Uwe Schindler [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cffa820 ] LUCENE-7651 : Fix Javadocs build for Java 8u121 by injecting "Google Code Prettify" without adding Javascript to Javadocs's -bottom parameter. Also update Prettify to latest version to fix Google Chrome issue. Conflicts: lucene/CHANGES.txt
        Hide
        thetaphi Uwe Schindler added a comment -

        I backported this change to Lucene 5.5.4, because otherwise we cannot run the build with Java 8 anymore. The problem is the following: Java 7's Javadocs do not add a "script.js" file to the Javadocs output, so this fix injects the javadocs, but because a "script.js" is nowhere referenced in the HTML files, prettyprint is not loaded.

        We have the following possibilities:

        • revert this backport commit, but it prevents smoketester from suceeding (as it checks also Java 8). It also makes it impossible to build the 5.5.4 release with Java 8
        • add more hacks to inject a script tag into the head element, but that's really complicated as you have to do it in every HTML file!
        • ignore the fact that Javadocs do not code-prettyprint correctly anymore in Java 7. The Javadocs are fine, just the code exaples are no longer syntax highlighted.

        I'd go for the third item. Any comments? If we go this route, I will add a comment to the Changelog that prettyprinting Javadocs is no longer working, if docs are build with Java 7.

        Show
        thetaphi Uwe Schindler added a comment - I backported this change to Lucene 5.5.4, because otherwise we cannot run the build with Java 8 anymore. The problem is the following: Java 7's Javadocs do not add a "script.js" file to the Javadocs output, so this fix injects the javadocs, but because a "script.js" is nowhere referenced in the HTML files, prettyprint is not loaded. We have the following possibilities: revert this backport commit, but it prevents smoketester from suceeding (as it checks also Java 8). It also makes it impossible to build the 5.5.4 release with Java 8 add more hacks to inject a script tag into the head element, but that's really complicated as you have to do it in every HTML file! ignore the fact that Javadocs do not code-prettyprint correctly anymore in Java 7. The Javadocs are fine, just the code exaples are no longer syntax highlighted. I'd go for the third item. Any comments? If we go this route, I will add a comment to the Changelog that prettyprinting Javadocs is no longer working, if docs are build with Java 7.
        Hide
        thetaphi Uwe Schindler added a comment -

        We don't have access to recent paid only Java 7 JDKs, but as the Javadocs fix was declared a security issue, I assume that it is also applied to Java 7, so without this fix Java 7 paid updates will fail the build, too.

        Show
        thetaphi Uwe Schindler added a comment - We don't have access to recent paid only Java 7 JDKs, but as the Javadocs fix was declared a security issue, I assume that it is also applied to Java 7, so without this fix Java 7 paid updates will fail the build, too.

          People

          • Assignee:
            thetaphi Uwe Schindler
            Reporter:
            thetaphi Uwe Schindler
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development