|
I don't think we need HADOOP_GLOBAL_OPTS, we can just use HADOOP_OPTS for that, but we could add a HADOOP_NAMENODE_OPTS that, when starting the namenode, is appended to HADOOP_OPTS, etc. In general, we could modify bin/hadoop to add the value of HADOOP_{$COMMAND}_OPTS to HADOOP_OPTS. Would that suffice?
On a first pass, that sounds like a very reasonable fix.
Can we just get rid of HADOOP_HEAPSIZE? If people want to set it, use the HADOOP_*_OPTS variables. I'm +1 for fixing this issue. +1 for separate heap size setting
Another approach, that also addresses
Setting Fix Version to Hadoop 0.17. It's important to remember that the hadoop.sh control files are dead stupid, and I don't think we should try to over-engineer them.
If you're willing to accept an unsupported solution, the bin/hadoop script happens to set the environment variable COMMAND before it sources hadoop-env.sh.
I prefer the first approach of using different variables. This would be easier to provision through HOD as well. Are there any specific advantages of using the second approach ? (I can see some, but still... smile)
What is the consensus? If there are no responses by tomorrow (Tuesday), will assume it the first approach (HADOOP-${COMMAND}-OPTS).
Attached patch handles the following env variables :
HADOOP_NAMENODE_OPTS Notes:
The default options are exactly same as before this patch. -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12379176/HADOOP-2551.patch  against trunk revision 643282. @author +1. The patch does not contain any @author tags. tests included -1. The patch doesn't appear to include any new or modified tests. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new javac compiler warnings. release audit +1. The applied patch does not generate any new release audit warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2128/testReport/ This message is automatically generated. Unit Tests : This only has simple changes to hadoop scripts.
Integrated in Hadoop-trunk #450 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/450/
) Release note added. IMHO I don't think this needs to be in the top level release notes.
Nigel's release note is better.
What happened with the idea of doing away with HADOOP_HEAPSIZE completely? The patch doesn't have any fix for this. Track this on another JIRA?
Currently, if I specify both HADOOP_HEAPSIZE=500 and HADOOP_JOBTRACKER_OPTS=-Xmx1024m, both get passed to jobtracker (JT command line: "java -Xmx500m -Xmx1024m .......") and the runtime picks up the last value. So, it works for now, but it would have been cleaner had HADOOP_HEAPSIZE been kicked of in entirety. Yes, HAEAPSIZE is not part of this jira. There was no mention of removing any existing variable.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
Improvement
Closed
Blocker
hadoop-env.sh needs finer granularity
HADOOP_GLOBAL_OPTS = applies to all processes
HADOOP_NAMENODE_OPTS = applies to just the namenode
HADOOP_TASK_OPTS = applies to just tasks
HADOOP_JT_OPTS = applies to the job tracker
HADOOP_TT_OPTS = applies to task trackers
HADOOP_CLIENT_OPTS = applies to clients, such as hadoop fsck, hadoop dfs, etc.
Additionally, it might be useful to split out the HADOOP_HEAPSIZE setting as well.