Update documentation for default configuration

• Key: HADOOP-4828
• Type: Improvement
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Sharad Agarwal
• Reporter: Sharad Agarwal
• Component/s: documentation
• Fix Version/s: 0.20.0

Description

Documentation needs to be updated as per the configuration changes in HADOOP-4631

Comments

[Amareshwari Sriramadasu]

Some comments:

1. The following files has references to hadoop-default and hadoop-site in the documentation.
src/core/overview.html
src/benchmarks/gridmix2/README.gridmix2

2. src/test, src/c++/libhdfs/tests/conf/ and src/contrib/test/ directories contains a file named hadoop-site.xml. I think this can be removed

3. The following files are still using hadoop-default and hadoop-site as the conf resources :
src/contrib/failmon/src/java/org/apache/hadoop/contrib/failmon/HDFSMerger.java
src/contrib/failmon/src/java/org/apache/hadoop/contrib/failmon/LocalStore.java
src/contrib/chukwa/src/java/org/apache/hadoop/chukwa/conf/ChukwaConfiguration.java
src/contrib/ec2/bin/hadoop-ec2-init-remote.sh
src/contrib/eclipse-plugin/src/java/org/apache/hadoop/eclipse/servers/RunOnHadoopWizard.java
src/contrib/hod

4. Also, the chinese docs have references in the following files:
src/docs/cn/src/documentation/content/xdocs/cluster_setup.xml
src/docs/cn/src/documentation/content/xdocs/hdfs_user_guide.xml
src/docs/cn/src/documentation/content/xdocs/site.xml
src/docs/cn/src/documentation/content/xdocs/streaming.xml

5. Hod documentation still has references to hadoop-site.xml

[Doug Cutting]

> src/test, src/c++/libhdfs/tests/conf/ and src/contrib/test/ directories contains a file named hadoop-site.xml. I think this can be removed

+1

[Sharad Agarwal]

The following files has references to hadoop-default and hadoop-site in the documentation. src/core/overview.html and src/benchmarks/gridmix2/README.gridmix2

could not find any references of hadoop-default/hadoop-site in src/core/overview.html. the current patch takes care of them.
src/benchmarks/gridmix2/README.gridmix2 contains references to hadoop-site.xml. I think it is fine as we are still supporting the hadoop-site.xml

src/test, src/c++/libhdfs/tests/conf/ and src/contrib/test/ directories contains a file named hadoop-site.xml. I think this can be removed

The reason to keep the empty hadoop-site.xml files is to avoid picking properties from conf/hadoop-site.xml. The test cases have conf also in classpath, so if an user has hadoop-site.xml in conf directory, the properties from it gets picked up. To override that bahaviour the empty hadoop-site.xml in test config folder is put. This would be removed once we remove the deprecated support for hadoop-site.xml.

The following files are still using hadoop-default and hadoop-site as the conf resources :

I think it is ok to refer hadoop-site.xml as we support it now. hadoop-default should not be referred. I found only following two to refer hadoop-default:
src/contrib/failmon/src/java/org/apache/hadoop/contrib/failmon/HDFSMerger.java
src/contrib/failmon/src/java/org/apache/hadoop/contrib/failmon/LocalStore.java
This should perhaps be fixed in another jira as it won't be a documentation change. btw it is a minor change and doesn't affect the functionality.

Also, the chinese docs have references in the following files:

Will file another jira for this.

Hod documentation still has references to hadoop-site.xml

Currently HOD generate the single configuration file as hadoop-site.xml and the framework supports that. So the documentation will still refer to it. This will be changed once HOD move to the new configuration structure.

[Amareshwari Sriramadasu]

could not find any references of hadoop-default/hadoop-site in src/core/overview.html. the current patch takes care of them.

All the href links in overview.html, have links to hadoop-default.
Line numbers - 137, 143, 226, 230, 234, 235, 242, 247, 249.
hadoop-site is referred in lines 154, 220

[Amareshwari Sriramadasu]

All the href links in overview.html, have links to hadoop-default.

Sorry. I missed the changes in overview.html in current patch. Please ignore my comment.

[Doug Cutting]

> The reason to keep the empty hadoop-site.xml files is to avoid picking properties from conf/hadoop-site.xml.

That sounds reasonable. We should then add a comment to src/test/hadoop-site.xml noting that it should be removed when support for hadoop-site.xml is removed.

[Sharad Agarwal]

added comment to src/test/hadoop-site.xml, src/contrib/test/hadoop-site.xml and src/c++/libhdfs/tests/conf/hadoop-site.xml as per Doug suggestion.

[Devaraj Das]

I just committed this. Thanks, Sharad!