[METRON-660] [Umbrella] up-to-date versioned documentation

• Type: Improvement
• Status: Done
• Priority: Major
• Resolution: Done
• Fix Version/s: 0.3.1
• Assignee: Matthew Foley
• Reporter: Matthew Foley
• Created: 16/Jan/17 20:17
• Updated: 07/Mar/17 20:35
• Resolved: 06/Feb/17 18:47

Description

We currently have three forms of documentation, with the following advantages and disadvantages:

Docs	Pro	Con
CWiki	Easy to edit, no special tools required, don't have to be a developer to contribute, google and wiki search	Not versioned, no review process, distant from the code, obsolete content tends to accumulate
Site	Versioned and reviewed, only committers can edit, google search	Yet another arcane toolset must be learned, only web programmers feel comfortable contributing, "asf-site" branch not related to code versions, distant from the code, tends to go obsolete due to non-maintenance
README.md	Versioned and reviewed, only committers can edit, tied to code versions, highly local to the code being documented	Non-developers don't know about them, may be scared by github, poor scoring in google search, no high-level presentation

Various discussion threads indicate the developer community likes README-based docs, and it's easy to see why from the above. I propose this extension to the README-based documentation, to address their disadvantages:

1. Produce a script that runs at build time and gathers the README.md files from all code subdirectories into a hierarchical tree. The script would have an exclusion list for non-user-content, which at this point would consist of [site/*, build_utils/*].
2. Utilize standard toolsets to publish the hierarchical collection of .md files as a "book", at least as far as the end user sees. Put the web gateway for this book in the public Metron Site, preferably with a "version" pull-down menu that allows access to older versions too.
3. Deprecate the use of cwiki for anything but long-lived demonstrations/tutorials that are unlikely to go obsolete. Move appropriate content from the cwiki into versioned .md files.

Extensive email discussion of these ideas occurred in thread https://mail-archives.apache.org/mod_mbox/incubator-metron-dev/201701.mbox/[e-mail]%3E

Issue Links

• is related to: METRON-716 Add README.md to site-book
• is related to: METRON-719 use of quadruple back-ticks in README.md file
• is related to: METRON-698 [site-book] Add link checker as a "unit test" for site-book build
• is related to: METRON-717 Move site.xml to .gitignore, and create site.xml.template as the tracked file
• relates to: METRON-720 modify generate-md.sh to re-throw errors from within 'find'
• relates to: METRON-718 integrate versioned site-book to public site (Metron home page)
• relates to: METRON-759 integrate javadoc to public site (Metron home page)
• links to: GitHub Pull Request #429

Sub-Tasks

• script to gather README.md files into a book-like hierarchy: Done
• feasibility POC for doxia-markdown plugin: Done
• [site-book] doxia-markdown plugin doesn't generate named anchors for H1 headings: Done