Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1086

hadoop commands in streaming tasks are trying to write to tasktracker's log

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1, 0.21.0, 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      This patch makes TT to set HADOOP_ROOT_LOGGER to INFO,TLA by default in the environment of taskjvm and its children.

      Description

      As HADOOP_ROOT_LOGGER is not set in the environment by TT for the children, the children of task jvm(in case of streaming) are trying to write to TT's log and getting the following Exception. Jobs are succeeded, but the issue is to be resolved by setting the environment variables by TT for use by children of task jvm in case of streaming job.

      When streaming calls hadoop commands, it's trying to write to TaskTracker log file.

      log4j:ERROR setFile(null,true) call failed.
      java.io.FileNotFoundException:
      /a/b/tasktracker.log (Permission denied)
      at java.io.FileOutputStream.openAppend(Native Method)
      at java.io.FileOutputStream.<init>(FileOutputStream.java:177)
      at java.io.FileOutputStream.<init>(FileOutputStream.java:102)
      at org.apache.log4j.FileAppender.setFile(FileAppender.java:290)
      at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:164)
      at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:216)
      at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
      at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
      at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
      at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
      at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
      at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:544)
      at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:440)
      at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:476)
      at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:471)
      at org.apache.log4j.LogManager.<clinit>(LogManager.java:125)
      at org.apache.log4j.Logger.getLogger(Logger.java:105)
      at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:229)
      at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:65)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
      at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529)
      at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235)
      at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:209)
      at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351)
      at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:138)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:57)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at org.apache.hadoop.fs.FsShell.main(FsShell.java:1880)
      log4j:ERROR Either File or DatePattern options are not set for appender [DRFA].

        Attachments

        1. MR-1086-yhadoop20.patch
          8 kB
          Ravi Gummadi
        2. MR-1086.v1.2.patch
          9 kB
          Ravi Gummadi
        3. MR-1086.v1.1.patch
          8 kB
          Ravi Gummadi
        4. MR-1086.v1.patch
          8 kB
          Ravi Gummadi
        5. MR-1086.patch
          8 kB
          Ravi Gummadi

          Activity

            People

            • Assignee:
              ravidotg Ravi Gummadi
              Reporter:
              ravidotg Ravi Gummadi
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: