Quick list of things:

• Remove reference to YARN_xyz where there is a HADOOP_xyz equivalent
• Remove the 'appender' env vars examples
• Remove HADOOP_SECURE_PID_DIR since hadoop-env.sh mentions it specifically. Plus that should be in the security documentation, not someone's first intro to hadoop.
• Mention that hadoop-env.sh can be used for all services and that yarn-env.sh and mapred-env.sh don't have to be used
• If HADOOP-10950 gets committed, mention that heapsizes can be in m, g, etc. units. Also no more HADOOP_HEAPSIZE, but use _MAX and _MIN
• Mention -D params for setting logging options too.
• Remove sbin commands for single-node start/stop. Use --daemon equivalents instead. (e.g., hadoop-daemon.sh start namenode => hdfs --daemon start namenode )

-00:

• initial version

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12686120/HADOOP-10908.patch
against trunk revision 03867eb.

+1 @author. The patch does not contain any @author tags.

+0 tests included. The patch appears to be a documentation patch that doesn't require tests.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. There were no new javadoc warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

-1 findbugs. The patch appears to introduce 111 new Findbugs (version 2.0.3) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5218//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5218//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5218//console

This message is automatically generated.

Nice work Allen Wittenauer!

+ with thousands of nodes. To play with Hadoop, you may first want to
+ install it on a single machine (see {{

Unknown macro: {./SingleCluster.html}

Single Node Setup}}) first.

We can remove the second 'first'.

size for the daemon. By default, the value is 1000. If you want to
configure the values separately for each deamon you can use.

That last sentence is confusing. It should probably provide an example of a daemon-specific HEAPSIZE variable

+ Start a HDFS DataNode with the following command on the
+ designated node as <hdfs>:

s/on the designated node/on each designated node/

+ If <<etc/hadoop/slaves>> and trusted access is configured, all of the
+ HDFS processes can be started with a utility script. As <hdfs>:

We should define what trusted access means. I assume you meant password-less SSH.

+[yarn]$ $HADOOP_YARN_HOME/bin/yarn --daemon stop nodemanager

---

+ If <<etc/hadoop/slaves>> and trusted access is configured, all of the
+ YARN processes can be started with a utility script. As <yarn>:
+
+----
+[yarn]$ $HADOOP_PREFIX/sbin/start-yarn.sh
+----
+
Stop the WebAppProxy server. Run on the WebAppProxy server as

This looks like a copy/paste error. It should say "YARN processes can be stopped" and "[yarn]$ $HADOOP_PREFIX/sbin/stop-yarn.sh".

-01:

• Fix Joey Echeverria's list of issues (I think) in various ways
• Fix some style issues
• Change HADOOP_HEAPSIZE to HADOOP_HEAPSIZE_MAX and mention unit support
• Fix some links
• Mention HADOOP_PREFIX as a global and using /etc/profile.d in an example
• Add pdsh as a package to add in place of rsync, since that's actually supported and useful

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12687769/HADOOP-10908-01.patch
against trunk revision bc21a1c.

+1 @author. The patch does not contain any @author tags.

+0 tests included. The patch appears to be a documentation patch that doesn't require tests.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. There were no new javadoc warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

-1 findbugs. The patch appears to introduce 3 new Findbugs (version 2.0.3) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

-1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

org.apache.hadoop.crypto.random.TestOsSecureRandom

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5289//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5289//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5289//console

This message is automatically generated.

-02:

• Added pointers to the wiki for Java recommendations
• Fixed some style/consistency issues
• Noted that mapred-env.sh and yarn-env.sh are optional
• More verbage around slaves
• Pointer to the new rack awareness documentation
• Clarified the ever growing hardware resource requirements
• Noted that JAVA_HOME is required (although it is auto-detected on OS X)
• Re-arranged some text so that one isn't configuring stuff after daemons are brought up.
• Documented some commands that were previously missing.

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12689866/HADOOP-10908-02.patch
against trunk revision e7257ac.

+1 @author. The patch does not contain any @author tags.

+0 tests included. The patch appears to be a documentation patch that doesn't require tests.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. There were no new javadoc warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

-1 findbugs. The patch appears to introduce 2 new Findbugs (version 2.0.3) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5359//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5359//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5359//console

This message is automatically generated.

-03:

• Major style fixes
• Generic header on command so that they can be referenced from other docs
• Reworked the text on a few subcommands due to reformatting

FYI, I know about the File System Shell docs missing the 's' for commands manual. Anything else anyone can see that needs to get changed/fixed?

+1

Thanks for updating the documentation, Alan!

-04:

• Fixes the missing s for the link. Woops.

Thanks Owen! I'll commit here in a sec.

Committed to trunk.

Thanks for the review!

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12690148/HADOOP-10908-03.patch
against trunk revision 562a701.

+1 @author. The patch does not contain any @author tags.

+0 tests included. The patch appears to be a documentation patch that doesn't require tests.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. There were no new javadoc warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

-1 findbugs. The patch appears to introduce 2 new Findbugs (version 2.0.3) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5365//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/5365//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5365//console

This message is automatically generated.

SUCCESS: Integrated in Hadoop-trunk-Commit #6810 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6810/)
HADOOP-10908. Common needs updates for shell rewrite (aw) (aw: rev 94d342e607e1db317bae7af86a34ae7cd3860348)

• hadoop-common-project/hadoop-common/CHANGES.txt
• hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/ClusterSetup.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/SingleCluster.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm

FAILURE: Integrated in Hadoop-Yarn-trunk #799 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/799/)
HADOOP-10908. Common needs updates for shell rewrite (aw) (aw: rev 94d342e607e1db317bae7af86a34ae7cd3860348)

• hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/SingleCluster.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/ClusterSetup.apt.vm
• hadoop-common-project/hadoop-common/CHANGES.txt

FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #65 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/65/)
HADOOP-10908. Common needs updates for shell rewrite (aw) (aw: rev 94d342e607e1db317bae7af86a34ae7cd3860348)

• hadoop-common-project/hadoop-common/src/site/apt/ClusterSetup.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
• hadoop-common-project/hadoop-common/CHANGES.txt
• hadoop-common-project/hadoop-common/src/site/apt/SingleCluster.apt.vm

FAILURE: Integrated in Hadoop-Hdfs-trunk #1997 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1997/)
HADOOP-10908. Common needs updates for shell rewrite (aw) (aw: rev 94d342e607e1db317bae7af86a34ae7cd3860348)

• hadoop-common-project/hadoop-common/src/site/apt/SingleCluster.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/ClusterSetup.apt.vm
• hadoop-common-project/hadoop-common/CHANGES.txt

SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #62 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/62/)
HADOOP-10908. Common needs updates for shell rewrite (aw) (aw: rev 94d342e607e1db317bae7af86a34ae7cd3860348)

• hadoop-common-project/hadoop-common/src/site/apt/ClusterSetup.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/SingleCluster.apt.vm
• hadoop-common-project/hadoop-common/CHANGES.txt
• hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm

FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #66 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/66/)
HADOOP-10908. Common needs updates for shell rewrite (aw) (aw: rev 94d342e607e1db317bae7af86a34ae7cd3860348)

• hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/ClusterSetup.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
• hadoop-common-project/hadoop-common/CHANGES.txt
• hadoop-common-project/hadoop-common/src/site/apt/SingleCluster.apt.vm

FAILURE: Integrated in Hadoop-Mapreduce-trunk #2016 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2016/)
HADOOP-10908. Common needs updates for shell rewrite (aw) (aw: rev 94d342e607e1db317bae7af86a34ae7cd3860348)

• hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/SingleCluster.apt.vm
• hadoop-common-project/hadoop-common/CHANGES.txt
• hadoop-common-project/hadoop-common/src/site/apt/ClusterSetup.apt.vm
• hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm