Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-812

Enabling app summary logs causes 'FileNotFound' errors

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta
    • Fix Version/s: 2.1.0-beta
    • Component/s: None
    • Labels:
      None

      Description

      RM app summary logs have been enabled as per the default config:

      #
      # Yarn ResourceManager Application Summary Log 
      #
      # Set the ResourceManager summary log filename
      yarn.server.resourcemanager.appsummary.log.file=rm-appsummary.log
      # Set the ResourceManager summary log level and appender
      yarn.server.resourcemanager.appsummary.logger=INFO,RMSUMMARY
      
      # Appender for ResourceManager Application Summary Log
      # Requires the following properties to be set
      #    - hadoop.log.dir (Hadoop Log directory)
      #    - yarn.server.resourcemanager.appsummary.log.file (resource manager app summary log filename)
      #    - yarn.server.resourcemanager.appsummary.logger (resource manager app summary log level and appender)
      
      log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary=${yarn.server.resourcemanager.appsummary.logger}
      log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary=false
      log4j.appender.RMSUMMARY=org.apache.log4j.RollingFileAppender
      log4j.appender.RMSUMMARY.File=${hadoop.log.dir}/${yarn.server.resourcemanager.appsummary.log.file}
      log4j.appender.RMSUMMARY.MaxFileSize=256MB
      log4j.appender.RMSUMMARY.MaxBackupIndex=20
      log4j.appender.RMSUMMARY.layout=org.apache.log4j.PatternLayout
      log4j.appender.RMSUMMARY.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
      

      This however, throws errors while running commands as non-superuser:

      -bash-4.1$ hadoop dfs -ls /
      DEPRECATED: Use of this script to execute hdfs command is deprecated.
      Instead use the hdfs command for it.
      
      log4j:ERROR setFile(null,true) call failed.
      java.io.FileNotFoundException: /var/log/hadoop/hadoopqa/rm-appsummary.log (No such file or directory)
              at java.io.FileOutputStream.openAppend(Native Method)
              at java.io.FileOutputStream.<init>(FileOutputStream.java:192)
              at java.io.FileOutputStream.<init>(FileOutputStream.java:116)
              at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
              at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)
              at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
              at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
              at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
              at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
              at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
              at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
              at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
              at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
              at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
              at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
              at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
              at org.apache.log4j.Logger.getLogger(Logger.java:104)
              at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:289)
              at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:109)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
              at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1116)
              at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:858)
              at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:604)
              at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:336)
              at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:310)
              at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685)
              at org.apache.hadoop.fs.FsShell.<clinit>(FsShell.java:41)
      Found 1 items
      drwxr-xr-x   - hadoop   hadoop            0 2013-06-12 21:28 /user
      

        Attachments

        1. YARN-812.2.txt
          3 kB
          Siddharth Seth
        2. YARN-812.txt
          0.9 kB
          Siddharth Seth

          Activity

            People

            • Assignee:
              sseth Siddharth Seth
              Reporter:
              rramya Ramya Sunil
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: