Solr
  1. Solr
  2. SOLR-2747

Include formatted Changes.html for release

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      Just like when releasing Lucene, Solr should also have a html formatted changes file.
      The Lucene Perl script (lucene/src/site/changes/changes2html.pl) should be reused.

      1. SOLR-2747_fix.patch
        0.7 kB
        Robert Muir
      2. SOLR-2747.patch
        44 kB
        Steve Rowe
      3. SOLR-2747.patch
        28 kB
        Steve Rowe
      4. SOLR-2747.patch
        27 kB
        Steve Rowe
      5. SOLR-2747.patch
        27 kB
        Steve Rowe
      6. SOLR-2747.patch
        4 kB
        Robert Muir

        Issue Links

          Activity

          Hide
          Martijn van Groningen added a comment -

          Removed 3.6 version.

          Show
          Martijn van Groningen added a comment - Removed 3.6 version.
          Hide
          Robert Muir added a comment -

          the start to a patch: but I only did the build piece here just as an experiment.

          Solr CHANGES.txt is formatted differently than Lucene's...

          Show
          Robert Muir added a comment - the start to a patch: but I only did the build piece here just as an experiment. Solr CHANGES.txt is formatted differently than Lucene's...
          Hide
          Robert Muir added a comment -

          The script in general seems to almost work already.

          I think its just a matter of some minor things, like filtering out
          some "---------------------- "'s in the Solr CHANGES, but i didnt look too hard.

          The reason i was looking at this is because I think its nice to link to an html
          copy of the full changes from the website release notes, e.g.

          http://lucene.apache.org/core/corenews.html

          Show
          Robert Muir added a comment - The script in general seems to almost work already. I think its just a matter of some minor things, like filtering out some "---------------------- "'s in the Solr CHANGES, but i didnt look too hard. The reason i was looking at this is because I think its nice to link to an html copy of the full changes from the website release notes, e.g. http://lucene.apache.org/core/corenews.html
          Hide
          Steve Rowe added a comment -

          Patch addressing the biggest conversion problems - probably 80% of the way there.

          Left to do:

          • Generate and check through all output to find problems.
          • Check that Lucene's Changes.html still gets produced properly

          Solr's CHANGES.txt does an issue grouping thing that's not done in Lucene's CHANGES.txt, e.g. the "SOLR-1873, SOLR-2358" (SolrCloud) entry under 4.0.0-ALPHA New Features. Attribution is screwed up (not colored green and not placed on its own line) for the main entry as well as for all subsidiary issues. Maybe CHANGES.txt should be edited to break issue groups up?

          Show
          Steve Rowe added a comment - Patch addressing the biggest conversion problems - probably 80% of the way there. Left to do: Generate and check through all output to find problems. Check that Lucene's Changes.html still gets produced properly Solr's CHANGES.txt does an issue grouping thing that's not done in Lucene's CHANGES.txt, e.g. the " SOLR-1873 , SOLR-2358 " (SolrCloud) entry under 4.0.0-ALPHA New Features. Attribution is screwed up (not colored green and not placed on its own line) for the main entry as well as for all subsidiary issues. Maybe CHANGES.txt should be edited to break issue groups up?
          Hide
          Steve Rowe added a comment -

          Oops, here's the correct patch.

          Show
          Steve Rowe added a comment - Oops, here's the correct patch.
          Hide
          Uwe Schindler added a comment -

          Can we also add some general overview page (maybe with the same XSL tricks and Pegdown on Solr, copied from Lucene)? This would make the Javadocs page easier and add some general information and a TOC. It could also link changes, too.

          I can work on that, maybe separate issue?

          Show
          Uwe Schindler added a comment - Can we also add some general overview page (maybe with the same XSL tricks and Pegdown on Solr, copied from Lucene)? This would make the Javadocs page easier and add some general information and a TOC. It could also link changes, too. I can work on that, maybe separate issue?
          Hide
          Uwe Schindler added a comment -
          Show
          Uwe Schindler added a comment - I refer to this page: https://builds.apache.org/job/Lucene-Artifacts-4.x/javadoc/
          Hide
          Robert Muir added a comment -

          FYI I tried to run the patch on lucene's changes but got:

               [exec] Not a HASH reference at /home/rmuir/workspace/lucene-trunk/lucene/site/changes/changes2html.pl line 742, <> line 7575.
          
          Show
          Robert Muir added a comment - FYI I tried to run the patch on lucene's changes but got: [exec] Not a HASH reference at /home/rmuir/workspace/lucene-trunk/lucene/site/changes/changes2html.pl line 742, <> line 7575.
          Hide
          Robert Muir added a comment -

          In case its useful (sorry, forgot to include):

          This is perl 5, version 14, subversion 2 (v5.14.2) built for x86_64-linux-gnu-thread-multi
          (with 53 registered patches, see perl -V for more detail)

          Show
          Robert Muir added a comment - In case its useful (sorry, forgot to include): This is perl 5, version 14, subversion 2 (v5.14.2) built for x86_64-linux-gnu-thread-multi (with 53 registered patches, see perl -V for more detail)
          Hide
          Steve Rowe added a comment - - edited

          FYI I tried to run the patch on lucene's changes but got

          Hmm, I don't get any errors like that. My perl:

          This is perl, v5.10.1 built for i686-cygwin-thread-multi-64int
          (with 13 registered patches, see perl -V for more detail)

          I got the same damned success on Debian Squeeze v6.0.5, where the perl version is (unfortunately) the same:

          This is perl, v5.10.1 built for x86_64-linux-gnu-thread-multi
          (with 56 registered patches, see perl -V for more detail)

          I see that the error you're getting is on line 742:

          739:  my $project_info_json = get_url_contents("$project_info_url/$product");
          740:
          741:  my $project_info = json2perl($project_info_json);
          742:  for my $version (@{$project_info->{versions}}) {
          

          The json2perl subroutine converts the JSON response from JIRA into a Perl-compatible serialized nested data structure, then eval's the result to reconstitute. Apparently the eval is failing for you - when eval fails, it returns the undefined value, which would cause the error you're getting when trying to treat an undefined value as a hash reference. Maybe JIRA isn't sending back the right thing?

          This version of the patch adds a script failure when eval fails, and prints out the eval error message, along with the munged JSON->Perl string that triggered the error - would you please try it, Robert?

          Show
          Steve Rowe added a comment - - edited FYI I tried to run the patch on lucene's changes but got Hmm, I don't get any errors like that. My perl: This is perl, v5.10.1 built for i686-cygwin-thread-multi-64int (with 13 registered patches, see perl -V for more detail) I got the same damned success on Debian Squeeze v6.0.5, where the perl version is (unfortunately) the same: This is perl, v5.10.1 built for x86_64-linux-gnu-thread-multi (with 56 registered patches, see perl -V for more detail) I see that the error you're getting is on line 742: 739: my $project_info_json = get_url_contents( "$project_info_url/$product" ); 740: 741: my $project_info = json2perl($project_info_json); 742: for my $version (@{$project_info->{versions}}) { The json2perl subroutine converts the JSON response from JIRA into a Perl-compatible serialized nested data structure, then eval 's the result to reconstitute. Apparently the eval is failing for you - when eval fails, it returns the undefined value, which would cause the error you're getting when trying to treat an undefined value as a hash reference. Maybe JIRA isn't sending back the right thing? This version of the patch adds a script failure when eval fails, and prints out the eval error message, along with the munged JSON->Perl string that triggered the error - would you please try it, Robert?
          Hide
          Steve Rowe added a comment -

          Crap, my svn was still generating the patch when I tried to attach it. Here's the full thing.

          Show
          Steve Rowe added a comment - Crap, my svn was still generating the patch when I tried to attach it. Here's the full thing.
          Hide
          Robert Muir added a comment -
               [exec] Can't use string ("") as a HASH ref while "strict refs" in use at /home/rmuir/workspace/lucene-trunk/lucene/site/changes/changes2html.pl line 742, <> line 7575.
          
          Show
          Robert Muir added a comment - [exec] Can't use string ("") as a HASH ref while "strict refs" in use at /home/rmuir/workspace/lucene-trunk/lucene/site/changes/changes2html.pl line 742, <> line 7575.
          Hide
          Robert Muir added a comment -

          Sorry this is more helpful, as i see a warning that might be related:

          Buildfile: /home/rmuir/workspace/lucene-trunk/lucene/build.xml
          
          changes-to-html:
              [mkdir] Created dir: /home/rmuir/workspace/lucene-trunk/lucene/build/docs/changes
               [exec] Use of uninitialized value $product in uc at /home/rmuir/workspace/lucene-trunk/lucene/site/changes/changes2html.pl line 47, <> line 7575.
               [exec] --2012-08-15 10:51:44--  https://issues.apache.org/jira/rest/api/2/project/
               [exec] Resolving issues.apache.org (issues.apache.org)... 140.211.11.121
               [exec] Connecting to issues.apache.org (issues.apache.org)|140.211.11.121|:443... connected.
               [exec] HTTP request sent, awaiting response... 200 OK
               [exec] Length: unspecified [application/json]
               [exec] Saving to: `STDOUT'
               [exec] 
               [exec]      0K .......... .......... .......... .......... .......... 18.8M
               [exec]     50K .......
               [exec] ... .......... .......... .......... ..........  540K
               [exec]    100K .......... .......... .                                8.62M=0.1s
               [exec] 
               [exec] 2012-08-15 10:51:44 (1.22 MB/s) - written to stdout [124503]
               [exec] 
               [exec] Can't use string ("") as a HASH ref while "strict refs" in use at /home/rmuir/workspace/lucene-trunk/lucene/site/changes/changes2html.pl line 742, <> line 7575.
          
          BUILD FAILED
          /home/rmuir/workspace/lucene-trunk/lucene/build.xml:484: The following error occurred while executing this line:
          /home/rmuir/workspace/lucene-trunk/lucene/common-build.xml:1768: exec returned: 255
          
          Total time: 1 second
          
          Show
          Robert Muir added a comment - Sorry this is more helpful, as i see a warning that might be related: Buildfile: /home/rmuir/workspace/lucene-trunk/lucene/build.xml changes-to-html: [mkdir] Created dir: /home/rmuir/workspace/lucene-trunk/lucene/build/docs/changes [exec] Use of uninitialized value $product in uc at /home/rmuir/workspace/lucene-trunk/lucene/site/changes/changes2html.pl line 47, <> line 7575. [exec] --2012-08-15 10:51:44-- https://issues.apache.org/jira/rest/api/2/project/ [exec] Resolving issues.apache.org (issues.apache.org)... 140.211.11.121 [exec] Connecting to issues.apache.org (issues.apache.org)|140.211.11.121|:443... connected. [exec] HTTP request sent, awaiting response... 200 OK [exec] Length: unspecified [application/json] [exec] Saving to: `STDOUT' [exec] [exec] 0K .......... .......... .......... .......... .......... 18.8M [exec] 50K ....... [exec] ... .......... .......... .......... .......... 540K [exec] 100K .......... .......... . 8.62M=0.1s [exec] [exec] 2012-08-15 10:51:44 (1.22 MB/s) - written to stdout [124503] [exec] [exec] Can't use string ("") as a HASH ref while "strict refs" in use at /home/rmuir/workspace/lucene-trunk/lucene/site/changes/changes2html.pl line 742, <> line 7575. BUILD FAILED /home/rmuir/workspace/lucene-trunk/lucene/build.xml:484: The following error occurred while executing this line: /home/rmuir/workspace/lucene-trunk/lucene/common-build.xml:1768: exec returned: 255 Total time: 1 second
          Hide
          Steve Rowe added a comment -

          I screwed up the last patch by not returning the results from the eval in the json2perl subroutine.

          Here's a fixed patch with more debug printing when things go wrong.

          Show
          Steve Rowe added a comment - I screwed up the last patch by not returning the results from the eval in the json2perl subroutine. Here's a fixed patch with more debug printing when things go wrong.
          Hide
          Robert Muir added a comment -

          Latest patch doesn't produce any errors for me with either lucene or solr.

          Show
          Robert Muir added a comment - Latest patch doesn't produce any errors for me with either lucene or solr.
          Hide
          Steve Rowe added a comment -

          Latest patch doesn't produce any errors for me with either lucene or solr.

          Cool, thanks for testing.

          I'll continue working on finishing the conversion.

          Let me know if you notice anything in particular that needs work.

          Show
          Steve Rowe added a comment - Latest patch doesn't produce any errors for me with either lucene or solr. Cool, thanks for testing. I'll continue working on finishing the conversion. Let me know if you notice anything in particular that needs work.
          Hide
          Steve Rowe added a comment -

          Patch (against branch_4x) - I think it's ready to commit.

          Show
          Steve Rowe added a comment - Patch (against branch_4x) - I think it's ready to commit.
          Hide
          Steve Rowe added a comment -

          Committed:

          Show
          Steve Rowe added a comment - Committed: branch_4x: r1378442 , r1378449 trunk: r1378452
          Hide
          Steve Rowe added a comment -

          Solr-Artifacts-trunk failed because changes2html.pl couldn't be found: from https://builds.apache.org/job/Solr-Artifacts-trunk/1958/:

          package-src-tgz:
               [exec] A    solr/build/solr/svn-export
          [...]
               [exec] A    solr/build/solr/svn-export/build.xml
               [exec] Exported revision 1378509.
             [delete] Deleting directory /usr/home/hudson/hudson-slave/workspace/Solr-Artifacts-trunk/solr/build/solr/svn-export/lucene/tools/javadoc/java6
             [delete] Deleting directory /usr/home/hudson/hudson-slave/workspace/Solr-Artifacts-trunk/solr/build/solr/svn-export/lucene/tools/clover
              [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Solr-Artifacts-trunk/solr/build/solr/svn-export/docs/changes
               [exec] Can't open perl script "/usr/home/hudson/hudson-slave/workspace/Solr-Artifacts-trunk/solr/build/solr/svn-export/site/changes/changes2html.pl": No such file or directory
          
          Show
          Steve Rowe added a comment - Solr-Artifacts-trunk failed because changes2html.pl couldn't be found: from https://builds.apache.org/job/Solr-Artifacts-trunk/1958/ : package-src-tgz: [exec] A solr/build/solr/svn-export [...] [exec] A solr/build/solr/svn-export/build.xml [exec] Exported revision 1378509. [delete] Deleting directory /usr/home/hudson/hudson-slave/workspace/Solr-Artifacts-trunk/solr/build/solr/svn-export/lucene/tools/javadoc/java6 [delete] Deleting directory /usr/home/hudson/hudson-slave/workspace/Solr-Artifacts-trunk/solr/build/solr/svn-export/lucene/tools/clover [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Solr-Artifacts-trunk/solr/build/solr/svn-export/docs/changes [exec] Can't open perl script "/usr/home/hudson/hudson-slave/workspace/Solr-Artifacts-trunk/solr/build/solr/svn-export/site/changes/changes2html.pl": No such file or directory
          Hide
          Robert Muir added a comment -

          the patch i sent to the ML:

          But we should decide where we want the output to go in the solr src dist, and also if we should run it on the lucene CHANGES too and consolidate it in some way.

          Show
          Robert Muir added a comment - the patch i sent to the ML: But we should decide where we want the output to go in the solr src dist, and also if we should run it on the lucene CHANGES too and consolidate it in some way.
          Hide
          Steve Rowe added a comment - - edited

          the patch i sent to the ML:

          Cool, thanks, I was able to successfully run ant prepare-release-no-sign under solr/ locally after applying your patch. I'll commit to branch_4x and trunk shortly.

          But we should decide where we want the output to go in the solr src dist, and also if we should run it on the lucene CHANGES too and consolidate it in some way.

          I think solr/docs/changes/ is fine for the target directory.

          AFAICT, no form of Lucene's CHANGES.txt has ever been included in a Solr release. I think it's a good idea, though. It could live at solr/docs/changes/Lucene-Changes.html.

          Along these lines, the Solr News page could include a link to Lucene's Changes.html alongside the link to the Solr Changes.html (I'm assuming that, as on the Lucene News page, the CHANGES.txt link will actually be to the formatted Changes.html, once the functionality provided by this issue has stabilized.)

          Show
          Steve Rowe added a comment - - edited the patch i sent to the ML: Cool, thanks, I was able to successfully run ant prepare-release-no-sign under solr/ locally after applying your patch. I'll commit to branch_4x and trunk shortly. But we should decide where we want the output to go in the solr src dist, and also if we should run it on the lucene CHANGES too and consolidate it in some way. I think solr/docs/changes/ is fine for the target directory. AFAICT, no form of Lucene's CHANGES.txt has ever been included in a Solr release. I think it's a good idea, though. It could live at solr/docs/changes/Lucene-Changes.html . Along these lines, the Solr News page could include a link to Lucene's Changes.html alongside the link to the Solr Changes.html (I'm assuming that, as on the Lucene News page, the CHANGES.txt link will actually be to the formatted Changes.html , once the functionality provided by this issue has stabilized.)
          Hide
          Steve Rowe added a comment -

          We can add Lucene Changes.html generation in a separate issue.

          Show
          Steve Rowe added a comment - We can add Lucene Changes.html generation in a separate issue.
          Hide
          Uwe Schindler added a comment -

          Closed after release.

          Show
          Uwe Schindler added a comment - Closed after release.

            People

            • Assignee:
              Steve Rowe
              Reporter:
              Martijn van Groningen
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development