Using confluence to manage the website content is attractive. However, it might also be nice to allow a less restrictive editing regime for non-website parts of the wiki (although some projects require an ICLA for all edits, see e.g. http://camel.apache.org/how-do-i-edit-the-website.html). Is this possible I wonder?

Here are some resources on generating websites from Confluence: http://incubator.apache.org/shiro/confluence-auto-export.html, https://cwiki.apache.org/CWIKI/.

Listing this as a blocker for first release.

Here's the incubator's podling site guidelines for explicit requirements:
http://incubator.apache.org/guides/sites.html

"Regardless of which tool you use, the web site should be maintained in the svn repository, and include the site generation tool as a binary file. This simplifies the process of site generation and enables changes to the site to be made by any committer. The generated site should also be checked into svn. This allows the generated site to be relocated to any part of the Apache site after incubation is complete."

"Using A Wiki To Create Documentation

Podlings may use a wiki to create documentation (including the website) providing that follow the guidelines. In particular, care must be taken to ensure that access to the wiki used to create documentation is restricted to only those with filed CLAs. The PPMC MUST review all changes and ensure that trust is not abused."

I'm not sure how access could be restricted - does anyone know? (cwiki/wiki support this?) Sounds like this might be painful vs contributors submitting doc patches through jira. Cassandra is recently out of incubation, they seem to have 2 static pages (home & download) and the rest of their docs are all wiki/javadoc based.

I'm not particularly fond of forrest after using it for hadoop/zookeeper. Any interest in using sphinx? (used to generate the docs for python). http://sphinx.pocoo.org/ (their site is itself generated with sphinx)

Thoughts?

> Cassandra is recently out of incubation, they seem to have 2 static pages (home & download) and the rest of their docs are all wiki/javadoc based.

They don't seem to have any documentation in the release artifacts either. That's another thing to consider - how to version documentation for a particular release (and whether it has to go into the tarball).

> I'm not particularly fond of forrest after using it for hadoop/zookeeper. Any interest in using sphinx? (used to generate the docs for python). http://sphinx.pocoo.org/ (their site is itself generated with sphinx)

I'm not a big Forrest fan either. The Java 5 thing is annoying, and I found it non-trivial to change the layout (e.g. to add a Google Analytics code). I haven't used Sphinx. AsciiDoc is yet another option I've used: http://www.methods.co.nz/asciidoc/.

They don't seem to have any documentation in the release artifacts either.

Actually they have the generated javadoc in the bin archive. Given that their primary audience is programmers this seems reasonable to me. They have some ops docs on the wiki but afaict they rely mostly on javadoc for developers and wiki for everything else.

The Java 5 thing is annoying,

I've heard some noise that forrest is working towards a release "rsn", as part of which the java 5 requirement would be dropped. However even disregarding the j5 issue I'm still have problems with forrest. It's very hard to debug, the configuration/filelayout is cryptic, and as you mention the page layout is nearly impossible to change.

I'm not set on any particular way of handling the site and the docs... just would be nice to pick something.

Asciidoc and sphinx both support pdf generation which is good.

Any idea what kind of site we would like? I'm more a function over form person.

Tom if you feel comfortable with asciidoc perhaps we should try that? Here's an example of the markup: http://www.methods.co.nz/asciidoc/asciidoc.txt
Looks ok to me. From the asciidoc web site if I understand correctly asciidoc can generate not only single page documents, but also "website" as well. Is that right? (multiple documents with navigation btw them?)

We can commit the site & doc source to trunk/src/docs/... We'd put the generated output into trunk/docs (committers responsible for updating as part of a doc commit) and the whirr website at apache along with new releases. That sound reasonable for a first pass? Or should we not have the generated docs in trunk? (I find it useful for ppl checking out the source, but it does add additional small burden to commiter).

I installed asciidoc on my system (karmic) and it did seem to pull in a number of dependencies... I notice avro also found this to be an issue:
https://issues.apache.org/jira/browse/AVRO-476
any concerns about this? Is everyone we expect to want to generate the docs just going to use a pkg mgr anyway? Another good reason to commit the generated docs into trunk (as I suggested in earlier comment)

Installing asciidoc pkg on karmic resulted in:

The following extra packages will be installed:
dblatex docbook-dsssl docbook-utils docbook-xsl docbook-xsl-doc-html jadetex latex-beamer latex-xcolor libostyle1c2 libsgmls-perl libsp1c2 openjade pgf preview-latex-style prosper ps2eps sgmlspl sp texlive texlive-fonts-recommended texlive-fonts-recommended-doc
texlive-generic-extra texlive-generic-recommended texlive-humanities texlive-humanities-doc texlive-latex-base texlive-latex-base-doc texlive-latex-extra texlive-latex-extra-doc texlive-latex-recommended texlive-latex-recommended-doc texlive-math-extra
texlive-pictures texlive-pictures-doc texlive-pstricks texlive-pstricks-doc tipa xmlto

> From the asciidoc web site if I understand correctly asciidoc can generate not only single page documents, but also "website" as well. Is that right? (multiple documents with navigation btw them?)

You can do this, but it involves transforming Asciidoc into DocBook first to generate a multi-page document/site. This is quite a complex toolchain (which is why you need so many packages), so I would prefer something simpler.

> Or should we not have the generated docs in trunk? (I find it useful for ppl checking out the source, but it does add additional small burden to commiter).

It is useful, but I think the reason we stopped doing this in Hadoop was because of the extra burden to generate Forrest docs. If the burden is small then I think we should check in the generated docs (this is why asciidoc is probably not a good choice).

I spent a bit more time looking at other Apache sites, and quite a few do use exported confluence to good effect (e.g. Camel, Jackrabbit, Sling, Wicket), but the limitation, as noted above, is that only people who have signed an ICLA can edit documentation. This sounds like a barrier to me, but perhaps I'm over optimistic about how much documentation is contributed? It's possible for anyone to add comments to a page though.

So I'm tending to think we should take the Cassandra route - have a minimal website (one page?) as a jumping off point, with versioned release documentation in javadoc (with top-level docs in the overview file - there would be one for core and one per service - or even extra plain HTML files, see http://java.sun.com/j2se/1.5.0/docs/tooldocs/solaris/javadoc.html#unprocessed), and a link to the wiki for further information. We could introduce a different documentation system at a later point if we wanted more structured documentation outside javadoc. This could be another confluence wiki, or sphinx (which looks easy to install, and the markup is reasonable), or something else (!).

Sound good?

It's possible to restrict particular pages on Confluence so they can only be created or edited by certain groups (e.g. those who have signed an ICLA). It would then be possible to publish only the restricted pages to the website. So I think we could use a single Confluence space for general users (who just need a Conflucence account) and ICLA users (for release docs and site) if we wanted to at some point.

I never paid much attention before but maven has site generation built in:
http://maven.apache.org/guides/mini/guide-site.html
what if we use this to build the site? example: http://incubator.apache.org/stonehenge/

we can use the apt format for our pages (wiki markup like format)
http://maven.apache.org/doxia/references/apt-format.html

We can put the mvn site pom & source content in subversion under incubator/whirr/site

that would leave the wiki free for ugc.

What do you think of that as an approach? I'm also fine with the "cassandra" approach if you still think that's the way to go, I only just noticed this mvn site generation capability but it seems like it would fit the bill.

or perhaps everything should just be in trunk, including the site/docs? tapestry seems to do this (notice the automatic deployment, including javadoc)
http://tapestry.formos.com/nightly/tapestry5/dev/checklist.html

+1 for using Maven site generation and for putting site and docs in trunk. Can we have a skin like Tapestry's or Maven's?

I noticed that Doxia (which is what Maven uses for content generation) supports a number of markup formats, including Confluence wiki markup, so we might use that one for consistency with our wiki (one fewer markup type to learn).

If we do this, I'll open up the wiki for editing by anyone with a confluence account.

We might use Tapestry's release checklist as a basis for Whirr's. Note in particular that they use Ant to build the release artifacts - I can understand this as I've had problems in the past getting Maven to build tarballs with the file layout I wanted.

Maven itself uses maven-stylus-skin, which would be fine. See http://svn.apache.org/repos/asf/maven/site/trunk/src/site/site.xml.

This post has a few useful tips about built-in reports: http://blog.xebia.com/2008/03/29/get-the-most-out-of-your-maven-reports/.

This is is my todo list still, but hasn't risen to the top. if someone wants to work on this please go ahead, I still plan to, just no time yet. Sorry.

First draft of the site.

untar into trunk

"mvn site" in trunk/site directory will generate trunk/site/target/site/index.html (etc...)

site.xml controls the basic layout of the site

Good start. Thanks for doing this. I would remove the empty pages and go with a smaller site to start with. We can add in content over time, and some is probably better on the wiki.

Also, you can fill in the committers' affiliations from http://wiki.apache.org/incubator/WhirrProposal.

We can use the maven-stylus-skin by adding the following section to site.xml below the version element:

  <skin>
    <groupId>org.apache.maven.skins</groupId>
    <artifactId>maven-stylus-skin</artifactId>
    <version>1.2</version>
  </skin>

I turned off the banner by adding this to resources/css/site.css:

#banner {
  height: 93px;
  background: none;
}

Please keep the comments coming - I'll try to work them all in during another round of hacking on this stuff, then I'd like to be in a position that ppl feel comfortable to commit the site skeleton. Which will allow everyone else to start submitting doc patches with actual docs (rather than just the setup stuff I'm doing now).

I would remove the empty pages and go with a smaller site to start with.

I think we should stick with boilerplate pages for the time being. here's why. by having the boilerplate (empty pages) it allows ppl to see what the site will look like when it's a bit more mature, also this acts as an "itch" for ppl to scratch - ie hopefully having a blank "serviceX" page will encourage devs to fill in, rather than just the page missing (out of sight out of mind). Also, we can drop the pages before a release if you feel strongly re the view outside development environment. Let me know if you feel strongly to go the other way.

> I think we should stick with boilerplate pages for the time being.

OK. We can drop them (or even fill them in) if they remain empty for too long.

Updated the site.tar.gz with latest:

1) updated the team affiliations
2) changed skin
3) removed faq and replaced with link to cwiki
we could start similar to cassandra: http://wiki.apache.org/cassandra/FAQ
4) added irc information

at this point I think we are good to go. let's try to get this committed asap at which point we can start filling in content and apply addl tweaks as necessary

made "roadmap" a link to the cwiki as well, I think this make more sense.

+1. Let's commit and continue to iterate on this before making it live.

Committed, wohoo!