Details
-
Type:
Improvement
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 0.21.0
-
Component/s: None
-
Labels:None
-
Hadoop Flags:Reviewed
-
Tags:hdfs datanode
Description
DataNode should use ToolRunner to parse generic hadoop options (like -Dconfig.var=value). Apart from being more consistent with rest of the hadoop programs, this is necessary to write a simple script to run multiple datanodes on the same node.
I had proposed a simple shell script to running multiple datanodes on a the user mailing list http://www.nabble.com/Contributing-to-hadoop-td22198897.html, but it requires that DataNode parses -D option.
I will submit a patch and a shell script that works.
Issue Links
- is related to
-
HDFS-439
HADOOP-5961 is incorrectly committed.
-
- Closed
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
Patch attached. DataNode does not need to implement 'Tool'. It can just use GenericOptionsParser directly.
This does not need unit tests.
+1 on the patch. Works as advertised. Is there any reason not to include the script and integrate it with the other shell scripts? The script works fine, but would need some validation (can only open 10 datanodes as it stands), header file, etc. This would make it easy for other developers to use without having to track down this JIRA months or years from now.
Thanks Jakob.
The current script is extremely basic, but gets the job done. It is more like command line alias. Still I spend more time the script than the actual code change in DataNode. It would take some time to write a good script and for associated help and support. We could consider later that if there is more interest.
run-datanode.sh : Minor typo fixes
Reattach the patch so that Hudson picks up the right one.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12409907/HADOOP-5961.patch
against trunk revision 781816.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/464/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/464/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/464/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/464/console
This message is automatically generated.
I just committed this.
Fix 'Fix Version'. Thanks Koji.
run-datanode.sh : an example script to start multiple datanodes on one machine.