Major Description
There are few issues with 'bin/yarn' and 'etc/hadoop/yarn-env.sh'. They are loosely related but fixes are minor and will go in the same files. Therefore I combined them into one JIRA.
Issues are:
1, bin/yarn has a dangling 'fi' in the last line. Thanks to shell for so compliant!
2. YARN_ROOT_LOGGER is defined as "INFO, DFRA" in yarn-env.sh. That's why 'bin/yarn' command doesn't show (by default) the log messages in client window. But when we used 'bin/hadoop', it shows the log correctly (because HADOOP_ROOT_LOOGER is "INFO,console" by default). Need to address this non-consistent behavior.
3. For each client command, yarn creates a log file in $YARN_LOG_DIR/yarn.log own by the 'end-user'. In a multi-tenant environment, the second user will not be able to create its own yarn.log in the same place causing the exception (pasted at the end). By default, it should write to $YARN_LOG_DIR/$USER/yarn.log instead.
Note:
--------
I plan to address only #1 and #2 in this JIRA. If we default the YARN_ROOT_LOGGER consistent with 'bin/hadoop', the issue #3 will not happen. The scope of this JIRA is come close to 'bin/hadoop' behavior.
Exception:
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /export/apps/hadoop/logs/yarn.log (Permission denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:142)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
at org.apache.log4j.Logger.getLogger(Logger.java:104)
at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:289)
at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:109)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1116)
at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:914)
at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:604)
at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:336)
at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:310)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685)
at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:165)
at org.apache.hadoop.util.RunJar.main(RunJar.java:158)
Attachments
Attachments
Issue Links
- is related to
-
YARN-1429 *nix: Allow a way for users to augment classpath of YARN daemons
-
- Closed
-