Lucene - Core
  1. Lucene - Core
  2. LUCENE-2894

Use of google-code-prettify for Lucene/Solr Javadoc

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1, 4.0-ALPHA
    • Component/s: general/javadocs
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      My company, RONDHUIT uses google-code-prettify (Apache License 2.0) in Javadoc for syntax highlighting:

      http://www.rondhuit-demo.com/RCSS/api/com/rondhuit/solr/analysis/JaReadingSynonymFilterFactory.html

      I think we can use it for Lucene javadoc (java sample code in overview.html etc) and Solr javadoc (Analyzer Factories etc) to improve or simplify our life.

      1. LUCENE-2894.patch
        35 kB
        Koji Sekiguchi
      2. LUCENE-2894.patch
        34 kB
        Koji Sekiguchi
      3. LUCENE-2894.patch
        36 kB
        Koji Sekiguchi
      4. LUCENE-2894.patch
        48 kB
        Koji Sekiguchi
      5. LUCENE-2894-solr-analysis-AtoC.patch
        12 kB
        Shinichiro Abe
      6. LUCENE-2894-solr-analysis-DtoH.patch
        24 kB
        Shinichiro Abe
      7. LUCENE-2894-solr-analysis-ItoN.patch
        14 kB
        Shinichiro Abe
      8. LUCENE-2894-solr-analysis-OtoR.patch
        16 kB
        Shinichiro Abe
      9. LUCENE-2894-solr-analysis-StoZ.patch
        17 kB
        Shinichiro Abe
      10. LUCENE-2894.patch
        30 kB
        Robert Muir
      11. LUCENE-2894.patch
        17 kB
        Robert Muir

        Activity

        Hide
        Grant Ingersoll added a comment -

        Bulk close for 3.1

        Show
        Grant Ingersoll added a comment - Bulk close for 3.1
        Hide
        Yonik Seeley added a comment -

        Do you mean Solr javadocs see the prettify stuff under lucene/ also?

        Yep, seems reasonable.

        Show
        Yonik Seeley added a comment - Do you mean Solr javadocs see the prettify stuff under lucene/ also? Yep, seems reasonable.
        Hide
        Koji Sekiguchi added a comment -

        in this case i would recommend moving the prettify stuff to under lucene/

        Do you mean Solr javadocs see the prettify stuff under lucene/ also?
        (Sorry if I missed something in community these days.)

        Show
        Koji Sekiguchi added a comment - in this case i would recommend moving the prettify stuff to under lucene/ Do you mean Solr javadocs see the prettify stuff under lucene/ also? (Sorry if I missed something in community these days.)
        Hide
        Robert Muir added a comment -

        setting to blocker, i don't think the prettify directory should be in dev-tools as it breaks the lucene-only build.

        dev-tools was just supposed to be some developer-only/maintained on as-is/optional stuff to help developers.

        in this case i would recommend moving the prettify stuff to under lucene/

        Show
        Robert Muir added a comment - setting to blocker, i don't think the prettify directory should be in dev-tools as it breaks the lucene-only build. dev-tools was just supposed to be some developer-only/maintained on as-is/optional stuff to help developers. in this case i would recommend moving the prettify stuff to under lucene/
        Hide
        Robert Muir added a comment -

        reopen as the lucene-only javadocs are broken

        Show
        Robert Muir added a comment - reopen as the lucene-only javadocs are broken
        Hide
        Robert Muir added a comment -

        attached is a patch that applies the stylesheet to some more code samples in various overview.html and package.html files.

        Show
        Robert Muir added a comment - attached is a patch that applies the stylesheet to some more code samples in various overview.html and package.html files.
        Hide
        Koji Sekiguchi added a comment -

        Sounds reasonable, thanks for reviewing and correcting them, Robert! Please commit the patch.

        Show
        Koji Sekiguchi added a comment - Sounds reasonable, thanks for reviewing and correcting them, Robert! Please commit the patch.
        Hide
        Robert Muir added a comment -

        some more improvements: the CollationKeyFilter's example uses KeywordTokenizer, since its usually for sorting.

        The various stemmers have standardtokenizer + lowercasefilter, most of these will not work correctly unless input is lowercased.

        The thaiwordfilter should not use n-gram tokenizer, because thaiwordfilter is word segmentation, so i changed it to use StandardTokenizer.

        Show
        Robert Muir added a comment - some more improvements: the CollationKeyFilter's example uses KeywordTokenizer, since its usually for sorting. The various stemmers have standardtokenizer + lowercasefilter, most of these will not work correctly unless input is lowercased. The thaiwordfilter should not use n-gram tokenizer, because thaiwordfilter is word segmentation, so i changed it to use StandardTokenizer.
        Hide
        Koji Sekiguchi added a comment -

        Committed O-to-R in r1074017 (trunk) and r1074023 (3x).

        Abe-san, ReversedWildcardFilterFactory should be applied only index time analysis, I've fixed your patch. Please see schema.xml for example. Thanks anyway for your great contribution!

        Show
        Koji Sekiguchi added a comment - Committed O-to-R in r1074017 (trunk) and r1074023 (3x). Abe-san, ReversedWildcardFilterFactory should be applied only index time analysis, I've fixed your patch. Please see schema.xml for example. Thanks anyway for your great contribution!
        Hide
        Koji Sekiguchi added a comment -

        Committed S-to-Z in r1074009 (trunk) and r1074011 (3x).

        Show
        Koji Sekiguchi added a comment - Committed S-to-Z in r1074009 (trunk) and r1074011 (3x).
        Hide
        Koji Sekiguchi added a comment -

        This is a really great start towards improving the documentation in this area.

        I agree.

        I've committed I-to-N in r1073806 (trunk) and r1073810 (3x). I'll check and commit the rest of patches tomorrow.
        Thanks again, Abe-san!

        Show
        Koji Sekiguchi added a comment - This is a really great start towards improving the documentation in this area. I agree. I've committed I-to-N in r1073806 (trunk) and r1073810 (3x). I'll check and commit the rest of patches tomorrow. Thanks again, Abe-san!
        Hide
        Robert Muir added a comment -

        Solr.analysis classes(I to Z) have been implemented by google-code-prettify.

        I just wanted to say thank you for adding these examples to all of the analysis factories!
        This is a really great start towards improving the documentation in this area.

        Show
        Robert Muir added a comment - Solr.analysis classes(I to Z) have been implemented by google-code-prettify. I just wanted to say thank you for adding these examples to all of the analysis factories! This is a really great start towards improving the documentation in this area.
        Hide
        Shinichiro Abe added a comment -

        Solr.analysis classes(I to Z) have been implemented by google-code-prettify.

        Show
        Shinichiro Abe added a comment - Solr.analysis classes(I to Z) have been implemented by google-code-prettify.
        Hide
        Koji Sekiguchi added a comment -

        solr.analysis classes(A to H) are implemented by google-code-prettify.Please confirm the patch.

        trunk: Committed revision 1073336.
        3x: Committed revision 1073344.

        Show
        Koji Sekiguchi added a comment - solr.analysis classes(A to H) are implemented by google-code-prettify.Please confirm the patch. trunk: Committed revision 1073336. 3x: Committed revision 1073344.
        Hide
        Koji Sekiguchi added a comment -

        Looks fine. Thanks, Abe-san!
        I'll look into them and commit tonight.

        Show
        Koji Sekiguchi added a comment - Looks fine. Thanks, Abe-san! I'll look into them and commit tonight.
        Hide
        Shinichiro Abe added a comment -

        solr.analysis classes(A to H) are implemented by google-code-prettify.Please confirm the patch.

        Show
        Shinichiro Abe added a comment - solr.analysis classes(A to H) are implemented by google-code-prettify.Please confirm the patch.
        Hide
        Koji Sekiguchi added a comment -
        Show
        Koji Sekiguchi added a comment - but Solr javadoc on hudson looks not good: https://hudson.apache.org/hudson/job/Solr-trunk/javadoc/org/apache/solr/handler/component/TermsComponent.html The problem was gone.
        Hide
        Steve Rowe added a comment -

        Both of the nightly Hudson Maven builds failed because javadoc jars were not produced by the Ant build (scroll down to the bottom to see the error about javadoc jars not being available to deploy):

        https://hudson.apache.org/hudson/job/Lucene-Solr-Maven-trunk/17/consoleText
        https://hudson.apache.org/hudson/job/Lucene-Solr-Maven-3.x/16/consoleText

        Show
        Steve Rowe added a comment - Both of the nightly Hudson Maven builds failed because javadoc jars were not produced by the Ant build (scroll down to the bottom to see the error about javadoc jars not being available to deploy): https://hudson.apache.org/hudson/job/Lucene-Solr-Maven-trunk/17/consoleText https://hudson.apache.org/hudson/job/Lucene-Solr-Maven-3.x/16/consoleText
        Hide
        Koji Sekiguchi added a comment -

        On my mac, there is prettify correctly under api directory after ant package:

        $ cd solr
        $ ant clean set-fsdir package
        $ ls build/docs/api/
        allclasses-frame.html          deprecated-list.html           package-list
        allclasses-noframe.html        help-doc.html                  prettify
        constant-values.html           index-all.html                 resources
        contrib-solr-analysis-extras   index.html                     serialized-form.html
        contrib-solr-cell              org                            solr
        contrib-solr-clustering        overview-frame.html            solrj
        contrib-solr-dataimporthandler overview-summary.html          stylesheet+prettify.css
        contrib-solr-uima              overview-tree.html
        
        Show
        Koji Sekiguchi added a comment - On my mac, there is prettify correctly under api directory after ant package: $ cd solr $ ant clean set-fsdir package $ ls build/docs/api/ allclasses-frame.html deprecated-list.html package -list allclasses-noframe.html help-doc.html prettify constant-values.html index-all.html resources contrib-solr-analysis-extras index.html serialized-form.html contrib-solr-cell org solr contrib-solr-clustering overview-frame.html solrj contrib-solr-dataimporthandler overview-summary.html stylesheet+prettify.css contrib-solr-uima overview-tree.html
        Hide
        Koji Sekiguchi added a comment -

        Reopening the issue.

        Lucene javadoc on hudson looks fine (syntax highlighting works correctly):

        https://hudson.apache.org/hudson/job/Lucene-trunk/javadoc/all/overview-summary.html

        but Solr javadoc on hudson looks not good:

        https://hudson.apache.org/hudson/job/Solr-trunk/javadoc/org/apache/solr/handler/component/TermsComponent.html

        Building of both javadocs on my local is working fine.

        Show
        Koji Sekiguchi added a comment - Reopening the issue. Lucene javadoc on hudson looks fine (syntax highlighting works correctly): https://hudson.apache.org/hudson/job/Lucene-trunk/javadoc/all/overview-summary.html but Solr javadoc on hudson looks not good: https://hudson.apache.org/hudson/job/Solr-trunk/javadoc/org/apache/solr/handler/component/TermsComponent.html Building of both javadocs on my local is working fine.
        Hide
        Koji Sekiguchi added a comment -

        trunk: Committed revision 1067551.
        3x: Committed revision 1067552.

        Show
        Koji Sekiguchi added a comment - trunk: Committed revision 1067551. 3x: Committed revision 1067552.
        Hide
        Koji Sekiguchi added a comment -

        New patch that supports lucene javadocs.

        I'd like to commit at the moment so that someone/volunteers can improve javadocs (e.g. TokenFilterFactories in Solr).

        Show
        Koji Sekiguchi added a comment - New patch that supports lucene javadocs. I'd like to commit at the moment so that someone/volunteers can improve javadocs (e.g. TokenFilterFactories in Solr).
        Hide
        Koji Sekiguchi added a comment -

        Support not only javadoc-core, but also javadoc-solrj, javadoc-all, javadoc-contrib. They should share a common prettify.

        Done in the attached patch.

        Show
        Koji Sekiguchi added a comment - Support not only javadoc-core, but also javadoc-solrj, javadoc-all, javadoc-contrib. They should share a common prettify. Done in the attached patch.
        Hide
        Koji Sekiguchi added a comment -

        TODO:

        • Support not only javadoc-core, but also javadoc-solrj, javadoc-all, javadoc-contrib. They should share a common prettify.
        • Support lucene javadocs.
        • Support modules javadocs?
        Show
        Koji Sekiguchi added a comment - TODO: Support not only javadoc-core, but also javadoc-solrj, javadoc-all, javadoc-contrib. They should share a common prettify. Support lucene javadocs. Support modules javadocs?
        Hide
        Koji Sekiguchi added a comment -

        New patch that uses "-header" option.

        Show
        Koji Sekiguchi added a comment - New patch that uses "-header" option.
        Hide
        Koji Sekiguchi added a comment -

        First draft for the review. Try ant clean javadoc-core under solr directory and see StopFilterFactory javadoc.

        In this draft, I used @

        {docRoot}

        (thanks Hoss!), and wrote the following lines in javadoc comment:

        <script src="{@docRoot}/prettify/prettify.js" type="text/javascript"></script>
        <script language="JavaScript">window.onload=function(){windowTitle();prettyPrint();}</script>
        

        but, they probably can be omitted if "-header" option as Hoss mentioned.

        Show
        Koji Sekiguchi added a comment - First draft for the review. Try ant clean javadoc-core under solr directory and see StopFilterFactory javadoc. In this draft, I used @ {docRoot} (thanks Hoss!), and wrote the following lines in javadoc comment: <script src= "{@docRoot}/prettify/prettify.js" type= "text/javascript" ></script> <script language= "JavaScript" >window.onload=function(){windowTitle();prettyPrint();}</script> but, they probably can be omitted if "-header" option as Hoss mentioned.
        Hide
        Hoss Man added a comment -

        could probably solve this in the general case using a combination of javadocs "-header" option, combined with the "

        {@docRoot}" macro, and putting the JS file in doc-files at the project root...

        http://download.oracle.com/javase/1.4.2/docs/tooldocs/windows/javadoc.html#header
        http://download.oracle.com/javase/1.4.2/docs/tooldocs/windows/javadoc.html#{@docRoot}

        http://download.oracle.com/javase/1.4.2/docs/tooldocs/windows/javadoc.html#unprocessed

        ...then it should be available in every generated HTML file

        Show
        Hoss Man added a comment - could probably solve this in the general case using a combination of javadocs "-header" option, combined with the " {@docRoot}" macro, and putting the JS file in doc-files at the project root... http://download.oracle.com/javase/1.4.2/docs/tooldocs/windows/javadoc.html#header http://download.oracle.com/javase/1.4.2/docs/tooldocs/windows/javadoc.html# {@docRoot} http://download.oracle.com/javase/1.4.2/docs/tooldocs/windows/javadoc.html#unprocessed ...then it should be available in every generated HTML file
        Hide
        Koji Sekiguchi added a comment -

        on a related note maybe this would encourage us to add more examples to our javadocs, which I think is really helpful for users.

        I agree.

        Any idea what it takes to integrate this with an ant task for javadocs?

        Nothing special at this moment. So the author of javadoc should know where prettify script is:

        <script src="../../../../prettify.js" type="text/javascript"></script>
        

        He should write it in another javadoc:

        <script src="../../../../../../../prettify.js" type="text/javascript"></script>
        

        I don't like it...

        (where prettify.js needs to be copied from somewhere (dev-tools? for example) to output dir of javadoc by ant task.)

        Show
        Koji Sekiguchi added a comment - on a related note maybe this would encourage us to add more examples to our javadocs, which I think is really helpful for users. I agree. Any idea what it takes to integrate this with an ant task for javadocs? Nothing special at this moment. So the author of javadoc should know where prettify script is: <script src= "../../../../prettify.js" type= "text/javascript" ></script> He should write it in another javadoc: <script src= "../../../../../../../prettify.js" type= "text/javascript" ></script> I don't like it... (where prettify.js needs to be copied from somewhere (dev-tools? for example) to output dir of javadoc by ant task.)
        Hide
        Robert Muir added a comment -

        I like this idea: on a related note maybe this would encourage us
        to add more examples to our javadocs, which I think is really helpful for users.

        Any idea what it takes to integrate this with an ant task for javadocs?

        Show
        Robert Muir added a comment - I like this idea: on a related note maybe this would encourage us to add more examples to our javadocs, which I think is really helpful for users. Any idea what it takes to integrate this with an ant task for javadocs?

          People

          • Assignee:
            Koji Sekiguchi
            Reporter:
            Koji Sekiguchi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development