Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5694

MR AM container syslog is empty

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.3.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When the property "mapreduce.task.userlog.limit.kb" is set non-zero in mapped-site.xml, AM container syslog remains empty. Without the log, it is hard to identify the cause of any MR AM failure.
      However, if "mapreduce.task.userlog.limit.kb" is not set (or defaulted to 0), syslog contents are correct.

      Bug details:
      For non zero "mapreduce.task.userlog.limit.kb", the code caches the log contents into memory, waited to be written to file when close() method is called at the end(Ref: ContainerLogAppender.java).
      Explicit call of LogManager.shutdown() from the main() (REF:MRAppMaster.java), ultimately call the *Appender.close().

      Root Cause:
      LogManager.shutdown() is not being called from MRAppMaster.java.
      Similar steps were taken correctly in YarnChild.java.

        Attachments

        1. MAPREDUCE-5694.1.patch
          1 kB
          Mohammad Islam

          Issue Links

            Activity

              People

              • Assignee:
                kamrul Mohammad Islam
                Reporter:
                kamrul Mohammad Islam
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: