Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5672

Provide optional RollingFileAppender for container log4j (syslog)

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.3.0
    • mr-am, mrv2
    • None
    • Reviewed

    Description

      This JIRA is an alternative take on YARN-1130

      We propose providing an option of using a RollingFileAppender(RFA)-based implementation of container log appender as means of log size control via mapreduce.task.userlog.limit.kb.

      The idea is to use mapreduce.task.userlog.limit.kb as maximumFileSize of RFA. In addition yarn.app.mapreduce.container.log.backups (task attempt containers) and yarn.app.mapreduce.am.log.backups (MR-AM) are passed as maxBackupIndex.

      Both current ContainerLogAppender (CLA) and new ContainerRollingLogAppender (CRLA) co-exist. CLA is the default. CRLA is chosen when mapreduce.task.userlog.limit.kb > 0 && *.backups > 0.

      Pros:
      1) CRLA output is visible in UI right away. CLA output with mapreduce.task.userlog.limit.kb > 0 is not visible until the task attempt finishes that prevents timely diagnostics.
      2) Even with excessive logging and a large mapreduce.task.userlog.limit.kb, no space is taken from the JVM heap.
      3) No UI impact, since YARN is already designed to deal with any log name beyond stderr/out, syslog, debug.out, profile.out

      Cons:
      1) if the logging is excessive there will be more local filesystem metadata I/O due to roll. That should be negligible in the grand scheme.

      Furthermore, to improve log consistency and completeness in the case of JVM crashes and SIGTERMing by NM, we propose to restore the MRv1 behavior of periodic log syncing (every 5s) and having log sync as part of a shutdown hook.

      Attachments

        1. Screen Shot 2013-12-05 at 3.23.33 PM.png
          318 kB
          Gera Shegalov
        2. Screen Shot 2013-12-05 at 3.21.02 PM.png
          56 kB
          Gera Shegalov
        3. MAPREDUCE-5672.v07.patch
          20 kB
          Jason Darrell Lowe
        4. MAPREDUCE-5672.v07.patch
          20 kB
          Gera Shegalov
        5. MAPREDUCE-5672.v06.patch
          19 kB
          Gera Shegalov
        6. MAPREDUCE-5672.v05.patch
          17 kB
          Gera Shegalov
        7. MAPREDUCE-5672.v04.patch
          27 kB
          Gera Shegalov
        8. MAPREDUCE-5672.v03.patch
          26 kB
          Gera Shegalov
        9. MAPREDUCE-5672.v02.patch
          26 kB
          Gera Shegalov
        10. MAPREDUCE-5672.v01.patch
          21 kB
          Gera Shegalov

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jira.shegalov Gera Shegalov Assign to me
            jira.shegalov Gera Shegalov
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment