Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-6871

Need a process for updating & maintaining the new quickstart tutorial (and any other tutorials added to the website)

    Details

    • Type: Task
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.3, 7.0
    • Component/s: None
    • Labels:
      None

      Description

      Prior to SOLR-6058 the /solr/tutorial.html link on the website contained only a simple landing page that then linked people to the "versioned" tutorial for the most recent release – or more specificly: the most recent release*s* (plural) when we were releasing off of multiple branches (ie: links to both the 4.0.0 tutorial, as well as the 3.6.3 tutorial when 4.0 came out)

      The old tutorial content lived along side the solr code, and was automatically branched, tagged & released along with Solr. When committing any changes to Solr code (or post.jar code, or the sample data, or the sample configs, etc..) you could also commit changes to the tutorial at th same time and be confident that it was clear what version of solr that tutorial went along with.

      As part of SOLR-6058, it seems that there was a concensus to move to a keeping "tutorial" content on the website, where it can be integrated directly in with other site content/navigation, and use the same look and feel.

      I have no objection to this in principle – but as a result of this choice, there are outstanding issues regarding how devs should go about maintaining this doc as changes are made to solr & the solr examples used in the tutorial.

      We need a clear process for where/how to edit the tutorial(s) as new versions of solr come out and cahnges are made that mandate corisponding hanges to the tutorial. this process should also account for things like having multiple versions of the tutorial live at one time (ie: at some point in the future, we'll certainly need to host the "5.13" tutorial if that's the current "stable" release, but we'll also want to host the tutorial for "6.0-BETA" so that people can try it out)

        Issue Links

          Activity

          Hide
          hossman Hoss Man added a comment -


          As a concrete example of why a process like this is crucial: there are a lot of changes going on righ now with bin/solr and what exactly happens when you run when "bin/solr -e foo" – and there is talk of adding a bin/post as well ... all of these things affect the quickstart tutorial, but folks making these changes can't edit the currently live quickstart.html file because it, by design, is suppose to be reflecting what happens if folks are trying out the examples in the "current release" (4.10.2)

          Show
          hossman Hoss Man added a comment - As a concrete example of why a process like this is crucial: there are a lot of changes going on righ now with bin/solr and what exactly happens when you run when "bin/solr -e foo" – and there is talk of adding a bin/post as well ... all of these things affect the quickstart tutorial, but folks making these changes can't edit the currently live quickstart.html file because it, by design, is suppose to be reflecting what happens if folks are trying out the examples in the "current release" (4.10.2)
          Hide
          hossman Hoss Man added a comment -

          ... folks making these changes can't edit the currently live quickstart.html file because it, by design, is suppose to be reflecting what happens if folks are trying out the examples in the "current release" (4.10.2)

          branching the website might be a way of addressing this type of "edits not ready to be live" but wouldn't help with the larger issue of having 2 versions of the tutorial "live" for users at the same time during periods when we are supporting multiple versions: ie: having a way for people to find the 4.10 quickstart content right after 5.0 is released – similar to how the live site's "tutorial.html" looked when 4.0 came out...

          https://svn.apache.org/viewvc/lucene/cms/trunk/content/solr/tutorial.mdtext?revision=1436785&view=markup&pathrev=1638603

          ...i know there has also been talk in other issues about wanting to include multiple tutorials beyond the "quickstart" (SOLR-6078, SOLR-6748, SOLR-6080, SOLR-6079, etc...) so all of those will have the same "versioning" issues as the current quickstart content.

          Show
          hossman Hoss Man added a comment - ... folks making these changes can't edit the currently live quickstart.html file because it, by design, is suppose to be reflecting what happens if folks are trying out the examples in the "current release" (4.10.2) branching the website might be a way of addressing this type of "edits not ready to be live" but wouldn't help with the larger issue of having 2 versions of the tutorial "live" for users at the same time during periods when we are supporting multiple versions: ie: having a way for people to find the 4.10 quickstart content right after 5.0 is released – similar to how the live site's "tutorial.html" looked when 4.0 came out... https://svn.apache.org/viewvc/lucene/cms/trunk/content/solr/tutorial.mdtext?revision=1436785&view=markup&pathrev=1638603 ...i know there has also been talk in other issues about wanting to include multiple tutorials beyond the "quickstart" ( SOLR-6078 , SOLR-6748 , SOLR-6080 , SOLR-6079 , etc...) so all of those will have the same "versioning" issues as the current quickstart content.
          Hide
          steve_rowe Steve Rowe added a comment -

          I think it should be possible to share Markdown content between the website quickstart tutorial and the versioned tutorial that ships with the release and is part of the versioned docs on the website. That should serve both purposes: a per-version tutorial, and the latest release tutorial with a consistent look&feel.

          I'll look at what it will take to get the doc build to do this. We already have some stuff getting converted from markdown now, via the pegdown macro (via groovy) in lucene/common-build.xml.

          Show
          steve_rowe Steve Rowe added a comment - I think it should be possible to share Markdown content between the website quickstart tutorial and the versioned tutorial that ships with the release and is part of the versioned docs on the website. That should serve both purposes: a per-version tutorial, and the latest release tutorial with a consistent look&feel. I'll look at what it will take to get the doc build to do this. We already have some stuff getting converted from markdown now, via the pegdown macro (via groovy) in lucene/common-build.xml .
          Hide
          jpountz Adrien Grand added a comment -

          There has been no activity for a long time so I decreased the priority.

          Show
          jpountz Adrien Grand added a comment - There has been no activity for a long time so I decreased the priority.
          Hide
          arafalov Alexandre Rafalovitch added a comment -

          Well, the shipped quickstart tutorial in Solr 6.0, says:

          To follow along with this tutorial, you will need…
          1. To meet the system requirements
          2. An Apache Solr release. This tutorial was written using Apache Solr 5.0.0

          So, I suspect it is not that the problem is less, but more that nobody quite knows where to start.

          Show
          arafalov Alexandre Rafalovitch added a comment - Well, the shipped quickstart tutorial in Solr 6.0, says: To follow along with this tutorial, you will need… 1. To meet the system requirements 2. An Apache Solr release. This tutorial was written using Apache Solr 5.0.0 So, I suspect it is not that the problem is less, but more that nobody quite knows where to start.
          Hide
          cpoerschke Christine Poerschke added a comment -

          Observations/Ideas from reading through http://lucene.apache.org/solr/quickstart.html today.

          Show
          cpoerschke Christine Poerschke added a comment - Observations/Ideas from reading through http://lucene.apache.org/solr/quickstart.html today. https://svn.apache.org/viewvc/lucene/cms/trunk/content/solr/quickstart.mdtext?revision=1705020 appears to be the source code but as mentioned above there is also in git (older version) https://github.com/apache/lucene-solr/blob/master/solr/site/quickstart.mdtext of the quickstart tutorial We could link the "release" part of the "... you will need ... An Apache Solr release." to the download page. ( line 13 ) The "Query tab" link appears to be broken. ( line 106 ) The "Transforming and Indexing Custom JSON data" link appears to be broken. ( line 192 ) The "Indexing JSON" heading could become "Indexing Solr JSON" and its paragraph link into the Solr JSON reference guide section as is done for Solr XML. ( line 169 ) The "Indexing CSV" heading could link into the reference guide also. ( line 194 )
          Hide
          arafalov Alexandre Rafalovitch added a comment -

          So, what do we do about this? Solr 6.2 is in the planning to be released and it would be good to get both website and the shipped version (still at '5.0') up to date and in sync.

          Is this just a matter of one person taking a lead and updating the tutorial as they see fit? Or is there some sort of discussion on which features need to be highlighted and which can be omitted.

          And with links to the Reference Guide, do we link into the live version of it (easy and convenient) or do an offline reference to the version-matched export?

          Show
          arafalov Alexandre Rafalovitch added a comment - So, what do we do about this? Solr 6.2 is in the planning to be released and it would be good to get both website and the shipped version (still at '5.0') up to date and in sync. Is this just a matter of one person taking a lead and updating the tutorial as they see fit? Or is there some sort of discussion on which features need to be highlighted and which can be omitted. And with links to the Reference Guide, do we link into the live version of it (easy and convenient) or do an offline reference to the version-matched export?
          Hide
          steve_rowe Steve Rowe added a comment -

          Patch updating the quickstart tutorial to 6.2.0 - the patch mentions but doesn't include a few screen capture images I modified to bring them up-to-date.

          I addressed all Christine Poerschke's suggestions above, except for:

          The "Indexing JSON" heading could become "Indexing Solr JSON" and its paragraph link into the Solr JSON reference guide section as is done for Solr XML.

          – I added a link to the ref guide section, but I left the heading as "Indexing JSON", since both and custom and Solr JSON are described there.

          I addressed the per-field search borkedness mentioned in SOLR-9526 by simply upcasing the search term name:foundation to name:Foundation - this works because strings fields can be successfully queried with exact full field values. I decided to leave in per-field querying, and not mention any caveats, since I figure SOLR-9526 will change the situation shortly. I didn't find any other problems.

          Finally, the patch also includes a new target generate-website-quickstart in solr/build.xml that makes few minor modifications to quickstart.mdtext (see comments in the build file for details) and puts the result at solr/build/website/quickstart.mdtext. This is a low-impact mechanism to allow the content to be shared between the bundled docs and the website.

          If there are no objections I'll commit in a day or so, and put the modified version of the quickstart tutorial up on the website.

          Show
          steve_rowe Steve Rowe added a comment - Patch updating the quickstart tutorial to 6.2.0 - the patch mentions but doesn't include a few screen capture images I modified to bring them up-to-date. I addressed all Christine Poerschke 's suggestions above, except for: The "Indexing JSON" heading could become "Indexing Solr JSON" and its paragraph link into the Solr JSON reference guide section as is done for Solr XML. – I added a link to the ref guide section, but I left the heading as "Indexing JSON", since both and custom and Solr JSON are described there. I addressed the per-field search borkedness mentioned in SOLR-9526 by simply upcasing the search term name:foundation to name:Foundation - this works because strings fields can be successfully queried with exact full field values. I decided to leave in per-field querying, and not mention any caveats, since I figure SOLR-9526 will change the situation shortly. I didn't find any other problems. Finally, the patch also includes a new target generate-website-quickstart in solr/build.xml that makes few minor modifications to quickstart.mdtext (see comments in the build file for details) and puts the result at solr/build/website/quickstart.mdtext . This is a low-impact mechanism to allow the content to be shared between the bundled docs and the website. If there are no objections I'll commit in a day or so, and put the modified version of the quickstart tutorial up on the website.
          Hide
          steve_rowe Steve Rowe added a comment -

          Alexandre Rafalovitch:

          So, what do we do about this? [...] Is this just a matter of one person taking a lead and updating the tutorial as they see fit? Or is there some sort of discussion on which features need to be highlighted and which can be omitted.

          I think the normal process should work okay - Lucene/Solr's official process is commit-then-review, but (as I've just done) pre-commit communication via patches should work too. But maybe you're thinking about the live website? I think we should err on the side of exhausting available review prior to publishing there. (My version of that here was giving a rough deadline at which I'll publish if I havn't gotten any review.)

          And with links to the Reference Guide, do we link into the live version of it (easy and convenient) or do an offline reference to the version-matched export?

          All of the links are to specific sections in the ref guide, and while I've seen internal PDF links work (depending on how the PDF was created and the PDF reader being used), I don't think they're dependable enough to allow us to use for this purpose. So I don't think there's a real choice at this point. However, once Cassandra Targett's ref guide work is in place (https://github.com/ctargett/refguide-asciidoc-poc), we should be able to make version-matched online docs that would be suitable for outbound links from the tutorial.

          Show
          steve_rowe Steve Rowe added a comment - Alexandre Rafalovitch : So, what do we do about this? [...] Is this just a matter of one person taking a lead and updating the tutorial as they see fit? Or is there some sort of discussion on which features need to be highlighted and which can be omitted. I think the normal process should work okay - Lucene/Solr's official process is commit-then-review, but (as I've just done) pre-commit communication via patches should work too. But maybe you're thinking about the live website? I think we should err on the side of exhausting available review prior to publishing there. (My version of that here was giving a rough deadline at which I'll publish if I havn't gotten any review.) And with links to the Reference Guide, do we link into the live version of it (easy and convenient) or do an offline reference to the version-matched export? All of the links are to specific sections in the ref guide, and while I've seen internal PDF links work (depending on how the PDF was created and the PDF reader being used), I don't think they're dependable enough to allow us to use for this purpose. So I don't think there's a real choice at this point. However, once Cassandra Targett 's ref guide work is in place ( https://github.com/ctargett/refguide-asciidoc-poc ), we should be able to make version-matched online docs that would be suitable for outbound links from the tutorial.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 8984e73660427b4be37f5eae177fbf8697e2e0c0 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8984e73 ]

          SOLR-6871: Updated the quickstart tutorial to cover the 6.2.0 release, and added ant target "generate-website-quickstart" to convert the bundled version of the tutorial into one suitable for the website.

          Show
          jira-bot ASF subversion and git services added a comment - Commit 8984e73660427b4be37f5eae177fbf8697e2e0c0 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8984e73 ] SOLR-6871 : Updated the quickstart tutorial to cover the 6.2.0 release, and added ant target "generate-website-quickstart" to convert the bundled version of the tutorial into one suitable for the website.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 53981795fd73e85aae1892c3c72344af7c57083a in lucene-solr's branch refs/heads/master from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5398179 ]

          SOLR-6871: Updated the quickstart tutorial to cover the 6.2.0 release, and added ant target "generate-website-quickstart" to convert the bundled version of the tutorial into one suitable for the website.

          Show
          jira-bot ASF subversion and git services added a comment - Commit 53981795fd73e85aae1892c3c72344af7c57083a in lucene-solr's branch refs/heads/master from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5398179 ] SOLR-6871 : Updated the quickstart tutorial to cover the 6.2.0 release, and added ant target "generate-website-quickstart" to convert the bundled version of the tutorial into one suitable for the website.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 1761836 from Steve Rowe in branch 'cms/trunk'
          [ https://svn.apache.org/r1761836 ]

          SOLR-6871: Updated the quickstart tutorial to cover the 6.2.0 release

          Show
          jira-bot ASF subversion and git services added a comment - Commit 1761836 from Steve Rowe in branch 'cms/trunk' [ https://svn.apache.org/r1761836 ] SOLR-6871 : Updated the quickstart tutorial to cover the 6.2.0 release
          Hide
          steve_rowe Steve Rowe added a comment -

          I committed the updated quickstart tutorial in both places. I'll leave the issue open for a little while in case anybody wants to do more here.

          Show
          steve_rowe Steve Rowe added a comment - I committed the updated quickstart tutorial in both places. I'll leave the issue open for a little while in case anybody wants to do more here.
          Hide
          janhoy Jan Høydahl added a comment -

          Precommit fails as noted by Dawid Weiss in SOLR-8186:

          -documentation-lint:
              [jtidy] Checking for broken html (such as invalid tags)...
             [delete] Deleting directory /mnt/storage/dweiss/work/lucene-solr/lucene/build/jtidy_tmp
               [echo] Checking for broken links...
               [exec]
               [exec] Crawl/parse...
               [exec]
               [exec] Verify...
               [exec]
               [exec] file:///mnt/storage/dweiss/work/lucene-solr/solr/build/docs/quickstart.html
               [exec]   BAD EXTERNAL LINK: http://lucene.apache.org/solr/downloads.html
               [exec]
               [exec] Broken javadocs links were found!
          

          Will commit a fix now

          Show
          janhoy Jan Høydahl added a comment - Precommit fails as noted by Dawid Weiss in SOLR-8186 : -documentation-lint: [jtidy] Checking for broken html (such as invalid tags)... [delete] Deleting directory /mnt/storage/dweiss/work/lucene-solr/lucene/build/jtidy_tmp [echo] Checking for broken links... [exec] [exec] Crawl/parse... [exec] [exec] Verify... [exec] [exec] file:///mnt/storage/dweiss/work/lucene-solr/solr/build/docs/quickstart.html [exec] BAD EXTERNAL LINK: http://lucene.apache.org/solr/downloads.html [exec] [exec] Broken javadocs links were found! Will commit a fix now
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit d14635445750cfcde2deca4d9f400d2c839f15eb in lucene-solr's branch refs/heads/master from Jan Høydahl
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d146354 ]

          SOLR-6871: Fix precommit - accept /solr/downloads.html as valid link

          Show
          jira-bot ASF subversion and git services added a comment - Commit d14635445750cfcde2deca4d9f400d2c839f15eb in lucene-solr's branch refs/heads/master from Jan Høydahl [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d146354 ] SOLR-6871 : Fix precommit - accept /solr/downloads.html as valid link
          Hide
          steve_rowe Steve Rowe added a comment -

          Thanks Jan Høydahl - for some reason your branch_6x commit didn't get posted here - from the email notification to commits@l.a.o:

          Repository: lucene-solr
          Updated Branches:
          refs/heads/branch_6x 082f8e3f9 -> 9611478c7

          SOLR-6871: Fix precommit - accept /solr/downloads.html as valid link

          (cherry picked from commit d146354)

          Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
          Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/9611478c
          Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/9611478c
          Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/9611478c

          Branch: refs/heads/branch_6x
          Commit: 9611478c7e249b7c65d3807e2ae672aabaefa50b
          Parents: 082f8e3
          Author: Jan Høydahl <janhoy@apache.org>
          Authored: Thu Sep 22 10:52:01 2016 +0200
          Committer: Jan Høydahl <janhoy@apache.org>
          Committed: Thu Sep 22 10:53:21 2016 +0200

          Show
          steve_rowe Steve Rowe added a comment - Thanks Jan Høydahl - for some reason your branch_6x commit didn't get posted here - from the email notification to commits@l.a.o: Repository: lucene-solr Updated Branches: refs/heads/branch_6x 082f8e3f9 -> 9611478c7 SOLR-6871 : Fix precommit - accept /solr/downloads.html as valid link (cherry picked from commit d146354) Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/9611478c Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/9611478c Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/9611478c Branch: refs/heads/branch_6x Commit: 9611478c7e249b7c65d3807e2ae672aabaefa50b Parents: 082f8e3 Author: Jan Høydahl <janhoy@apache.org> Authored: Thu Sep 22 10:52:01 2016 +0200 Committer: Jan Høydahl <janhoy@apache.org> Committed: Thu Sep 22 10:53:21 2016 +0200
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          Closing after 6.3.0 release.

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - Closing after 6.3.0 release.

            People

            • Assignee:
              steve_rowe Steve Rowe
              Reporter:
              hossman Hoss Man
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development