Infrastructure
  1. Infrastructure
  2. INFRA-5637

Migrate Hadoop's site to CMS or svnpubsub

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: Initial Clearing
    • Component/s: SvnPubSub
    • Labels:
      None

      Description

      This JIRA is to start the process of migrating Hadoop's website to either the CMS or svnpubsub. Thanks in advance for your help with this.

        Activity

        Aaron T. Myers created issue -
        Show
        #asfinfra IRC Bot added a comment - <danielsh> http://www.apache.org/dev/infra-contact#what-we-need-to-know
        #asfinfra IRC Bot made changes -
        Field Original Value New Value
        Status Waiting for Infra [ 10011 ] Waiting for user [ 10010 ]
        Hide
        Aaron T. Myers added a comment -
        Seems like the easiest thing for Hadoop to do will be to move to svnpubsub. Given that, I think the "SVN URL of the compiled site" is http://svn.apache.org/repos/asf/hadoop/common/site/main/publish
        Show
        Aaron T. Myers added a comment - Seems like the easiest thing for Hadoop to do will be to move to svnpubsub. Given that, I think the "SVN URL of the compiled site" is http://svn.apache.org/repos/asf/hadoop/common/site/main/publish
        Aaron T. Myers made changes -
        Status Waiting for user [ 10010 ] Waiting for Infra [ 10011 ]
        Hide
        Joe Schaefer added a comment -
        Migrating an ant-based site to the CMS is very straightforward: see http://www.apache.org/dev/cmsadoption#ant
        for details. All that would change notionally is that you'd be giving up local builds and subsequent checkins of
        built content in favor of CI-based builds of you content. You also get a basic webgui to make simple editing
        painless, but at the end it's up to you.

        Let me know which direction you prefer: svnpubsub or CMS and I will proceed.
        Show
        Joe Schaefer added a comment - Migrating an ant-based site to the CMS is very straightforward: see http://www.apache.org/dev/cmsadoption#ant for details. All that would change notionally is that you'd be giving up local builds and subsequent checkins of built content in favor of CI-based builds of you content. You also get a basic webgui to make simple editing painless, but at the end it's up to you. Let me know which direction you prefer: svnpubsub or CMS and I will proceed.
        Joe Schaefer made changes -
        Assignee Joe Schaefer [ joes ]
        Hide
        Aaron T. Myers added a comment -
        I don't feel super strongly about it, but it seems like going with svnpubsub would actually be less painful for Hadoop. We're already used to rebuilding the site ourselves, and we're actually a Maven-based site these days, not ant.

        Suresh/Tom/Todd/Stack - do any of you have opinions on the subject?

        Thanks a lot for your help with this, Joe.
        Show
        Aaron T. Myers added a comment - I don't feel super strongly about it, but it seems like going with svnpubsub would actually be less painful for Hadoop. We're already used to rebuilding the site ourselves, and we're actually a Maven-based site these days, not ant. Suresh/Tom/Todd/Stack - do any of you have opinions on the subject? Thanks a lot for your help with this, Joe.
        Hide
        stack added a comment -
        [~atm] I agree svnpubsub seems least disruptive. My opinion could change after hbase migration though. There I am open to going maven route but by my current level of understanding, it would require rejiggering of hard-won site generation multi-module config... so am leaning toward svnpubsub there too.
        Show
        stack added a comment - [~atm] I agree svnpubsub seems least disruptive. My opinion could change after hbase migration though. There I am open to going maven route but by my current level of understanding, it would require rejiggering of hard-won site generation multi-module config... so am leaning toward svnpubsub there too.
        Hide
        Thomas Graves added a comment -
        I don't have a strong opinion, but I haven't had time to learn about each in detail yet.
        Show
        Thomas Graves added a comment - I don't have a strong opinion, but I haven't had time to learn about each in detail yet.
        Hide
        Thomas Graves added a comment -
        After learning a bit more about them, I agree with Aaron and stack, svnpubsub seems least painful at this point
        Show
        Thomas Graves added a comment - After learning a bit more about them, I agree with Aaron and stack, svnpubsub seems least painful at this point
        Gavin made changes -
        Component/s SvnPubSub [ 12313605 ]
        Component/s Website [ 11708 ]
        Gavin made changes -
        Assignee Joe Schaefer [ joes ] Gavin [ ipv6guru ]
        Joe Schaefer made changes -
        Assignee Gavin [ ipv6guru ] Joe Schaefer [ joes ]
        Hide
        Joe Schaefer added a comment -
        Ping. Is there an svn tree ready for us to configure your site with?
        FYI: on Feb 1 we will be disabling the rsync jobs so it's important to
        get this issue settled soonish.
        Show
        Joe Schaefer added a comment - Ping. Is there an svn tree ready for us to configure your site with? FYI: on Feb 1 we will be disabling the rsync jobs so it's important to get this issue settled soonish.
        Joe Schaefer made changes -
        Status Waiting for Infra [ 10011 ] Waiting for user [ 10010 ]
        Hide
        Aaron T. Myers added a comment -
        Hi Joe,

        I believe you can use this SVN tree for this purpose: http://svn.apache.org/repos/asf/hadoop/common/site/main/publish

        Is there any way we can test this before going live with it? Thanks a lot for your help with all of this.
        Show
        Aaron T. Myers added a comment - Hi Joe, I believe you can use this SVN tree for this purpose: http://svn.apache.org/repos/asf/hadoop/common/site/main/publish Is there any way we can test this before going live with it? Thanks a lot for your help with all of this.
        Aaron T. Myers made changes -
        Status Waiting for user [ 10010 ] Waiting for Infra [ 10011 ]
        Hide
        Thomas Graves added a comment -
        So as Joe stated we can't publish updates anymore (as of Feb 1). What is the status and eta on this?
        Show
        Thomas Graves added a comment - So as Joe stated we can't publish updates anymore (as of Feb 1). What is the status and eta on this?
        Hide
        Joe Schaefer added a comment -
        All set.
        Show
        Joe Schaefer added a comment - All set.
        Joe Schaefer made changes -
        Status Waiting for Infra [ 10011 ] Closed [ 6 ]
        Resolution Fixed [ 1 ]
        Hide
        Thomas Graves added a comment -
        Thanks Joe, so how long does it take to make whats in svn repo available at hadoop.apache.org? I'm not seeing the changes checked in be present. For instance http://hadoop.apache.org/releases.html doesn't show updates in http://svn.apache.org/repos/asf/hadoop/common/site/main/publish/releases.html
        Show
        Thomas Graves added a comment - Thanks Joe, so how long does it take to make whats in svn repo available at hadoop.apache.org? I'm not seeing the changes checked in be present. For instance http://hadoop.apache.org/releases.html doesn't show updates in http://svn.apache.org/repos/asf/hadoop/common/site/main/publish/releases.html
        Hide
        Thomas Graves added a comment -
        Show
        Thomas Graves added a comment - so it appears to be picking up what is published under http://svn.apache.org/viewvc/hadoop/common/site/common/publish/ and not http://svn.apache.org/repos/asf/hadoop/common/site/main/publish
        Hide
        Doug Cutting added a comment -
        Just found this issue. The links to the documentation were all 404'ing this morning. None of the linked docs were in svn. I pruned the list of links down to just the current releases and added these docs to svn. Someone should update the HowToRelease docs to say that docs should no longer be copied directly to /www/hadoop.apache.org on people but rather committed to the site in svn.
        Show
        Doug Cutting added a comment - Just found this issue. The links to the documentation were all 404'ing this morning. None of the linked docs were in svn. I pruned the list of links down to just the current releases and added these docs to svn. Someone should update the HowToRelease docs to say that docs should no longer be copied directly to /www/hadoop.apache.org on people but rather committed to the site in svn.
        Hide
        Thomas Graves added a comment -
        Thanks Doug. How did you change the location svnpubsub was picking up from?
        I've updated the HowToReleasePostMavenization. I'll ping Matt to see if he can update the HowToRelease.
        Show
        Thomas Graves added a comment - Thanks Doug. How did you change the location svnpubsub was picking up from? I've updated the HowToReleasePostMavenization. I'll ping Matt to see if he can update the HowToRelease.
        Hide
        Doug Cutting added a comment -
        > How did you change the location svnpubsub was picking up from?

        I didn't. I think someone in infra must have switched it before I came along. Through trial and error I determined that http://svn.apache.org/repos/asf/hadoop/common/site/main/publish was what corresponded to the live site. Once I figured this out I removed the site/{hdfs,mapred,common} directories to avoid future confusion. I'm 90% certain that there's nothing there we need any longer. If I'm wrong then we can easily resurrect them. I hope I didn't step on anyone's toes. I figured with the docs 404'ing it was more important just to fix it than to first have a discussion on a list and reach consensus. We can still do that, of course.

        I'm not 100% confident that we don't need to resurrect the docs for some older releases. Previously I think we had the docs for every Hadoop release ever still on the website. Perhaps we should look at the logs and see if folks are regularly accessing docs for some older releases?
        Show
        Doug Cutting added a comment - > How did you change the location svnpubsub was picking up from? I didn't. I think someone in infra must have switched it before I came along. Through trial and error I determined that http://svn.apache.org/repos/asf/hadoop/common/site/main/publish was what corresponded to the live site. Once I figured this out I removed the site/{hdfs,mapred,common} directories to avoid future confusion. I'm 90% certain that there's nothing there we need any longer. If I'm wrong then we can easily resurrect them. I hope I didn't step on anyone's toes. I figured with the docs 404'ing it was more important just to fix it than to first have a discussion on a list and reach consensus. We can still do that, of course. I'm not 100% confident that we don't need to resurrect the docs for some older releases. Previously I think we had the docs for every Hadoop release ever still on the website. Perhaps we should look at the logs and see if folks are regularly accessing docs for some older releases?
        Gavin made changes -
        Fix Version/s Initial Clearing [ 12325964 ]

          People

          • Assignee:
            Joe Schaefer
            Reporter:
            Aaron T. Myers
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development