Details

    • Type: Task Task
    • Status: In Progress
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      This is an umbrella JIRA to achieve the migration of Apache James web sites to the Apache CMS as requested by Infra.

      Benefits of using CMS:

      • continuous deployment of web-site: edit & commit -> stage -> publish
      • easy website deploy to production and staging - just clicks or script run
      • central management of James website pages for all projects - improved consistency across sub-projects
      • easy content management from console (using svn/git ) or browser (via bookmarklet)
      • allow non-committers to contribute to the website much more easily

      References:

      How to use Apache CMS web interface: http://www.youtube.com/watch?v=xcDZN3Lu6HA
      How to use Apache CMS as a anonymous (non-committer): http://www.youtube.com/watch?v=7fvg1pfHLhE

        Issue Links

          Activity

          Hide
          Eric Charles added a comment -

          This is the inital information

          From: Joe Schaefer <joe_schaefer@yahoo.com>
          To: Apache Infrastructure <infrastructure@apache.org>
          Cc:
          Date: Wed, 8 Feb 2012 04:26:48 -0800 (PST)
          Subject: Mandatory svnpubsub migration by Jan 2013
          [PLEASE DO NOT RESPOND TO THIS POST! DIRECT ALL FURTHER
          INQUIRIES TO infrastructure@apache.org]

          FYI: infrastructure policy regarding website hosting has
          changed as of November 2011: we are requiring all websites
          and dist/ dirs to be svnpubsub or ASF CMS backed by the end of 2012.
          If your PMC has already met this requirement congratulations,
          you can ignore the remainder of this post.

          As stated on http://www.apache.org/dev/project-site.html#svnpubsub
          we are migrating our webserver infrastructure to 100% svnpubsub
          over the course of 2012. If your site does not currently make
          use of this technology, it is time to consider a migration effort,
          as rsync-based sites will be PERMANENTLY FROZEN in Jan 2013 due

          to infra disabling the hourly rsync jobs. While we recommend
          migrating to the ASF CMS [0] for Anakia based or Confluence based
          sites, and have provided tooling [1] to help facilitate this,
          we are only mandating svnpubsub (which the CMS uses itself).

          svnpubsub is a client-server system whereby a client watches an
          svn working copy for relevant commit notifications from the svn
          server. It subsequently runs svn up on the working copy, bringing
          in the relevant changes. sites that use static build technologies
          that commit the build results to svn are naturally compatible with
          svnpubsub; simply file a JIRA ticket with INFRA to request a
          migration: any commits to the resulting build tree will be

          instantly picked up on the live site.

          The CMS is a more elaborate system based on svnpubsub which
          provides a webgui for convenient online editing. Dozens of
          sites have already successfully deployed using the CMS and
          are quite happy with the results. The system is sufficiently
          flexible to accommodate a wide variety of choices regarding
          templating systems and storage formats, but most sites have
          standardized on the combination of Django and Markdown. Talk
          to infra if you would like to use the CMS in this or some other
          fashion, we'll see what we can do.

          NOTE: the policy for dist/ dirs for managing project releases is
          similar. We have setup a dedicated svn server for handling this,
          please contact infra when you are ready to start using it.

          HTH

          [0]: http://www.apache.org/dev/cms
          [1]: https://svn.apache.org/repos/infra/websites/cms/conversion-utilities/

          Show
          Eric Charles added a comment - This is the inital information From: Joe Schaefer <joe_schaefer@yahoo.com> To: Apache Infrastructure <infrastructure@apache.org> Cc: Date: Wed, 8 Feb 2012 04:26:48 -0800 (PST) Subject: Mandatory svnpubsub migration by Jan 2013 [PLEASE DO NOT RESPOND TO THIS POST! DIRECT ALL FURTHER INQUIRIES TO infrastructure@apache.org] FYI: infrastructure policy regarding website hosting has changed as of November 2011: we are requiring all websites and dist/ dirs to be svnpubsub or ASF CMS backed by the end of 2012. If your PMC has already met this requirement congratulations, you can ignore the remainder of this post. As stated on http://www.apache.org/dev/project-site.html#svnpubsub we are migrating our webserver infrastructure to 100% svnpubsub over the course of 2012. If your site does not currently make use of this technology, it is time to consider a migration effort, as rsync-based sites will be PERMANENTLY FROZEN in Jan 2013 due to infra disabling the hourly rsync jobs. While we recommend migrating to the ASF CMS [0] for Anakia based or Confluence based sites, and have provided tooling [1] to help facilitate this, we are only mandating svnpubsub (which the CMS uses itself). svnpubsub is a client-server system whereby a client watches an svn working copy for relevant commit notifications from the svn server. It subsequently runs svn up on the working copy, bringing in the relevant changes. sites that use static build technologies that commit the build results to svn are naturally compatible with svnpubsub; simply file a JIRA ticket with INFRA to request a migration: any commits to the resulting build tree will be instantly picked up on the live site. The CMS is a more elaborate system based on svnpubsub which provides a webgui for convenient online editing. Dozens of sites have already successfully deployed using the CMS and are quite happy with the results. The system is sufficiently flexible to accommodate a wide variety of choices regarding templating systems and storage formats, but most sites have standardized on the combination of Django and Markdown. Talk to infra if you would like to use the CMS in this or some other fashion, we'll see what we can do. NOTE: the policy for dist/ dirs for managing project releases is similar. We have setup a dedicated svn server for handling this, please contact infra when you are ready to start using it. HTH [0] : http://www.apache.org/dev/cms [1] : https://svn.apache.org/repos/infra/websites/cms/conversion-utilities/
          Show
          Eric Charles added a comment - Avro seems to have found its way https://issues.apache.org/jira/browse/AVRO-1025?focusedCommentId=13204156&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13204156
          Hide
          Stefano Bagnara added a comment -

          I have no time right now, but maybe it worth trying if this works:
          http://babyloncandle.blogspot.com/2009/04/deploying-maven-artifacts-to-googlecode.html

          (Does Avro use a maven site or forrest site?)

          Show
          Stefano Bagnara added a comment - I have no time right now, but maybe it worth trying if this works: http://babyloncandle.blogspot.com/2009/04/deploying-maven-artifacts-to-googlecode.html (Does Avro use a maven site or forrest site?)
          Hide
          Eric Charles added a comment -

          Thx Stefano for the link. I will take time to read it and to figure out how we can handle this new requirement.
          Help appreciated !

          mmh seems like avro uses forrest.

          Show
          Eric Charles added a comment - Thx Stefano for the link. I will take time to read it and to figure out how we can handle this new requirement. Help appreciated ! mmh seems like avro uses forrest.
          Hide
          Eric Charles added a comment -

          Stefano, about http://babyloncandle.blogspot.com/2009/04/deploying-maven-artifacts-to-googlecode.html , the webdav option has been recently discussed on the maven mailing list (http://markmail.org/message/ryytdbhy534c7z3q) in // to the thread on infra.

          Show
          Eric Charles added a comment - Stefano, about http://babyloncandle.blogspot.com/2009/04/deploying-maven-artifacts-to-googlecode.html , the webdav option has been recently discussed on the maven mailing list ( http://markmail.org/message/ryytdbhy534c7z3q ) in // to the thread on infra.
          Hide
          Stefano Bagnara added a comment -

          Ouch. Thatnk you for investigating this!

          We have 10 months before ASF will switch off the current rsync and maven TLP is in our same shoes and they have better skills at maven.. so I think we should wait a bit and look after them!
          And I really hope we don't have to use custom shell scripts on the user side to deal with site deployment (we use java/maven to avoid dealing with cross platforms issues and then we don't like to be forced using shell commands, right?)

          Show
          Stefano Bagnara added a comment - Ouch. Thatnk you for investigating this! We have 10 months before ASF will switch off the current rsync and maven TLP is in our same shoes and they have better skills at maven.. so I think we should wait a bit and look after them! And I really hope we don't have to use custom shell scripts on the user side to deal with site deployment (we use java/maven to avoid dealing with cross platforms issues and then we don't like to be forced using shell commands, right?)
          Hide
          Eric Charles added a comment -

          Yes, 10 months is much, and also not so much.
          Indeed, the best for now is to monitor maven guys' initiative and hopefully get a cross-platform/minimal-impact solution for free.

          Show
          Eric Charles added a comment - Yes, 10 months is much, and also not so much. Indeed, the best for now is to monitor maven guys' initiative and hopefully get a cross-platform/minimal-impact solution for free.
          Hide
          Eric Charles added a comment -

          See also "Implement svnpubsub using "mvn site" ?" thread on infra mailing list.

          Show
          Eric Charles added a comment - See also "Implement svnpubsub using "mvn site" ?" thread on infra mailing list.
          Show
          Eric Charles added a comment - More info on cms integration with maven, read: http://maven.40175.n5.nabble.com/Maven-site-and-CMS-svnpubsub-td5529649.html 'Maven' section on http://www.staging.apache.org/dev/cmsadoption.html https://issues.apache.org/jira/browse/INFRA-4466
          Hide
          Ioan Eugen Stan added a comment -

          Started reading about svnpubsub and cms and trying to come up with a good solution. I prefer something on Markdown for supplying content because it's text, easy to write/read and can generate the usual formats: html, pdf. I will also check Apache Forrest.

          Is there anything else I should have in mind?

          Show
          Ioan Eugen Stan added a comment - Started reading about svnpubsub and cms and trying to come up with a good solution. I prefer something on Markdown for supplying content because it's text, easy to write/read and can generate the usual formats: html, pdf. I will also check Apache Forrest. Is there anything else I should have in mind?
          Hide
          Ioan Eugen Stan added a comment -

          Revision: 1309389
          Author: ieugen
          Date: 2012-04-04 16:54
          Message: Issue #JAMES-1374 - Migrate James web sites to Apache CMS
          JAMES-1374

          • initial import for James site with Apache CMS
          • we can use svn-externals to put other svn versioned documentation files here
          Show
          Ioan Eugen Stan added a comment - Revision: 1309389 Author: ieugen Date: 2012-04-04 16:54 Message: Issue # JAMES-1374 - Migrate James web sites to Apache CMS JAMES-1374 initial import for James site with Apache CMS we can use svn-externals to put other svn versioned documentation files here
          Hide
          Ioan Eugen Stan added a comment -

          James staging website is set-up at: https://svn.apache.org/repos/infra/websites/staging/james/trunk/

          Every change to site-cms will make a site rebuild and the generated HTML files will be available at the above URL. You can pull the pages to view them locally and you can also build them locally. To view the staging site go to http://james.staging.apache.org/ .

          Now we have to find a good site template and create our web site. We can view the build status of the build and publish it from here https://cms.apache.org/. I personally like the one from http://openejb.apache.org/ but I'm open to suggestions and contributions since web design is not my thing.

          Cheers,

          Show
          Ioan Eugen Stan added a comment - James staging website is set-up at: https://svn.apache.org/repos/infra/websites/staging/james/trunk/ Every change to site-cms will make a site rebuild and the generated HTML files will be available at the above URL. You can pull the pages to view them locally and you can also build them locally. To view the staging site go to http://james.staging.apache.org/ . Now we have to find a good site template and create our web site. We can view the build status of the build and publish it from here https://cms.apache.org/ . I personally like the one from http://openejb.apache.org/ but I'm open to suggestions and contributions since web design is not my thing. Cheers,
          Hide
          Stefano Bagnara added a comment -

          Why don't you use the same template we used in the maven skin? I'm not against changing the html "theme" but mixing things doesn't sound as a good plan: this issue is about using Apache CMS, not altering our skin/logo/layout.

          Show
          Stefano Bagnara added a comment - Why don't you use the same template we used in the maven skin? I'm not against changing the html "theme" but mixing things doesn't sound as a good plan: this issue is about using Apache CMS, not altering our skin/logo/layout.
          Hide
          Eric Charles added a comment -

          See also JAMES-1401 as solution for now (once all sub-sites will be checked-in, I will ask infra the publication via svnpubsub).

          Show
          Eric Charles added a comment - See also JAMES-1401 as solution for now (once all sub-sites will be checked-in, I will ask infra the publication via svnpubsub).
          Hide
          Ioan Eugen Stan added a comment -

          Hi Eric, Stefano,

          We can probably use the current site skin but we will have to change the build scripts for the site in order to do that and this will take some investigation and some grunt work. Currently Apache CMS uses perl scripts to build the site and expects the django-style templates. Any new features => hack the scripts [1]. I don't feel like hacking perl. Tried it and it still haunts me.

          Eric, note that we can use svn externals to link the current sites to site-cms. The open-ejb guys link examples from trunk like this.

          I have also tested the bookmarklet online editing solution and it's brilliant. Please try it [2] on any apache page and if you don't love it I don't know what to say. You will see why I strongly recommend changing stuff to CMS.

          p.s. busy these days

          Take care,

          [1] http://www.apache.org/dev/cms.html#templating
          [2] https://cms.apache.org/#bookmark

          Show
          Ioan Eugen Stan added a comment - Hi Eric, Stefano, We can probably use the current site skin but we will have to change the build scripts for the site in order to do that and this will take some investigation and some grunt work. Currently Apache CMS uses perl scripts to build the site and expects the django-style templates. Any new features => hack the scripts [1] . I don't feel like hacking perl. Tried it and it still haunts me. Eric, note that we can use svn externals to link the current sites to site-cms. The open-ejb guys link examples from trunk like this. I have also tested the bookmarklet online editing solution and it's brilliant. Please try it [2] on any apache page and if you don't love it I don't know what to say. You will see why I strongly recommend changing stuff to CMS. p.s. busy these days Take care, [1] http://www.apache.org/dev/cms.html#templating [2] https://cms.apache.org/#bookmark
          Hide
          Eric Charles added a comment -

          As previously commented, further to maven monitoring, I found commits in a sandbox:
          http://mail-archives.apache.org/mod_mbox/maven-commits/201203.mbox/%3C20120301011459.BBB3C238889B@eris.apache.org%3E

          and now seems available on maven website : http://maven.apache.org/sandbox/plugins/asf-svnpubsub-plugin/

          Will be worth to investigate/test.

          Show
          Eric Charles added a comment - As previously commented, further to maven monitoring, I found commits in a sandbox: http://mail-archives.apache.org/mod_mbox/maven-commits/201203.mbox/%3C20120301011459.BBB3C238889B@eris.apache.org%3E and now seems available on maven website : http://maven.apache.org/sandbox/plugins/asf-svnpubsub-plugin/ Will be worth to investigate/test.
          Hide
          Ioan Eugen Stan added a comment -

          Script which automates the mapping of legacy forrest documentation to markdown?

          http://forrest.apache.org/pluginDocs/plugins_0_80/org.apache.forrest.plugin.output.Markdown/

          I think it's good for us too.

          Show
          Ioan Eugen Stan added a comment - Script which automates the mapping of legacy forrest documentation to markdown? http://forrest.apache.org/pluginDocs/plugins_0_80/org.apache.forrest.plugin.output.Markdown/ I think it's good for us too.
          Hide
          Eric Charles added a comment -

          No sure if forrest file format are 100% compatible with maven ones.
          To be tested and used as inspiration if we decide to automate the format transformation.

          Show
          Eric Charles added a comment - No sure if forrest file format are 100% compatible with maven ones. To be tested and used as inspiration if we decide to automate the format transformation.
          Hide
          Ioan Eugen Stan added a comment -

          HI,

          I've moved the site building to maven so now we are using Apache CMS with maven site building (a.k.a cms external build). This means that we can move the site to CMS infrastructure with little difficulty and no visual changes. You can see the results here: http://james.staging.apache.org/index.html .

          I've migrated the server site for now and in another location, just to see how things are rendered, you can find a list of URL's here: https://svn.apache.org/repos/infra/websites/staging/james/trunk/content/ to check out page rendering (ex. http://james.staging.apache.org/config-users.html).

          This being said, there are a few things to do:

          We will have to move the all xdocs to site-cms/trunk/content/xdoc/

          {path on web server}

          so they will get rendered in the same place. For example server/xdoc folder should be moved to content/xdoc/server (or server/3 I'm a bit confused) so the paths will build correctly.
          We will also have to link the pages back to the projects via svn:externals to have the docs with the modules. By doing this we will trigger a commit to site-cms when we are updating the component documentation and the site will get built automatically (in about 4 min).

          This seems like a perfect solution so I hope there are no other problems on the way. I would really like to see this through so we can concentrate more on improving stuff.

          Show
          Ioan Eugen Stan added a comment - HI, I've moved the site building to maven so now we are using Apache CMS with maven site building (a.k.a cms external build). This means that we can move the site to CMS infrastructure with little difficulty and no visual changes. You can see the results here: http://james.staging.apache.org/index.html . I've migrated the server site for now and in another location, just to see how things are rendered, you can find a list of URL's here: https://svn.apache.org/repos/infra/websites/staging/james/trunk/content/ to check out page rendering (ex. http://james.staging.apache.org/config-users.html ). This being said, there are a few things to do: We will have to move the all xdocs to site-cms/trunk/content/xdoc/ {path on web server} so they will get rendered in the same place. For example server/xdoc folder should be moved to content/xdoc/server (or server/3 I'm a bit confused) so the paths will build correctly. We will also have to link the pages back to the projects via svn:externals to have the docs with the modules. By doing this we will trigger a commit to site-cms when we are updating the component documentation and the site will get built automatically (in about 4 min). This seems like a perfect solution so I hope there are no other problems on the way. I would really like to see this through so we can concentrate more on improving stuff.
          Hide
          Eric Charles added a comment -

          Hi Ioan,
          How does the editing process happen?: does it edit the body only, or the complete page (body + menu).
          We should only be able to edit the content, and be able to regenerate the site applying any skin and any menu structure.
          Am I missing something?
          Thx, Eric

          Show
          Eric Charles added a comment - Hi Ioan, How does the editing process happen?: does it edit the body only, or the complete page (body + menu). We should only be able to edit the content, and be able to regenerate the site applying any skin and any menu structure. Am I missing something? Thx, Eric
          Hide
          Ioan Eugen Stan added a comment -

          Well,

          Apache CMS is just continuous integration for web-sites. Buildbot tracks changes in our repo and issues mvn site commands to build the site. It then copies the site from target/whatever to the staging repo (svn working copy??). From there it can go to live site via publishing.

          So site is just a maven project that builds docs with everithing that comes with this: you can do all that maven can with the speed (slowness) that comes with it. To build the site locally just svn co https://svn.apache.org/repos/asf/james/site-cms/trunk/ && mvn site. WHat you get is what gets to staging and publishing.

          You can also edit the pages via browser (tried it, but had to clean the working copy first via update link because I was getting mdtext files), then commit from browser and wait about 3 min for build.

          Buildbot shows the status for builds at http://ci.apache.org/builders/james-site-staging (staging). There should be some other urls there too.

          I hope this answers your question.
          Bye,

          Show
          Ioan Eugen Stan added a comment - Well, Apache CMS is just continuous integration for web-sites. Buildbot tracks changes in our repo and issues mvn site commands to build the site. It then copies the site from target/whatever to the staging repo (svn working copy??). From there it can go to live site via publishing. So site is just a maven project that builds docs with everithing that comes with this: you can do all that maven can with the speed (slowness) that comes with it. To build the site locally just svn co https://svn.apache.org/repos/asf/james/site-cms/trunk/ && mvn site. WHat you get is what gets to staging and publishing. You can also edit the pages via browser (tried it, but had to clean the working copy first via update link because I was getting mdtext files), then commit from browser and wait about 3 min for build. Buildbot shows the status for builds at http://ci.apache.org/builders/james-site-staging (staging). There should be some other urls there too. I hope this answers your question. Bye,
          Hide
          Ioan Eugen Stan added a comment -

          I think this is also useful, part of a conversation between me and Joe from infra:

          > I didn't exactly get how to use extpaths.txt. It will track files
          > after they are deleted from svn? Could you please better explain this?

          There are 3 trees in play in the CMS: the source tree in the asf/james/site,
          the staging tree in infra/websites/staging/james and the production tree
          in infra/websites/production/james. extpaths.txt lists paths to preserve
          in the production tree, after they are deleted from the source tree and then
          subsequently deleted from the staging tree by buildbot.

          Show
          Ioan Eugen Stan added a comment - I think this is also useful, part of a conversation between me and Joe from infra: > I didn't exactly get how to use extpaths.txt. It will track files > after they are deleted from svn? Could you please better explain this? There are 3 trees in play in the CMS: the source tree in the asf/james/site, the staging tree in infra/websites/staging/james and the production tree in infra/websites/production/james. extpaths.txt lists paths to preserve in the production tree, after they are deleted from the source tree and then subsequently deleted from the staging tree by buildbot.
          Hide
          Eric Charles added a comment -

          So we can still change the xml file and commit them, It will be published.

          You say: "You can also edit the pages via browser (tried it, but had to clean the working copy first via update link because I was getting mdtext files), then commit from browser and wait about 3 min for build."

          --> how can I try? I would like to see the difference compared to the current svnpubsub (functionality, speed,...)
          Thx, Eric

          Show
          Eric Charles added a comment - So we can still change the xml file and commit them, It will be published. You say: "You can also edit the pages via browser (tried it, but had to clean the working copy first via update link because I was getting mdtext files), then commit from browser and wait about 3 min for build." --> how can I try? I would like to see the difference compared to the current svnpubsub (functionality, speed,...) Thx, Eric

            People

            • Assignee:
              Ioan Eugen Stan
              Reporter:
              Eric Charles
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Development