Flume
  1. Flume
  2. FLUME-1073

Default Log4j configuration file should have a rolling policy

    Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.1.0
    • Fix Version/s: v1.2.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      If the user keeps the default log4j.properties in place, flume now logs to flume.log instead of stdout. This can be overridden on the command line by specifying an argument similar to the following: -Dflume.root.logger=DEBUG,console

      Description

      Flume Agents are expected to run for indefinite amount of time and will likely generate a lot of information via it's logs. As such, we need to have a log rolling policy in place to ensure that the system does not run out of disk space due to the log file sizes.

        Issue Links

          Activity

          Hide
          Arvind Prabhakar added a comment -

          Suggested configuration:

          log4j.rootLogger=INFO,LOGFILE
          log4j.appender.LOGFILE=org.apache.log4j.RollingFileAppender
          log4j.appender.LOGFILE.MaxFileSize=10MB
          log4j.appender.LOGFILE.MaxBackupIndex=10
          log4j.appender.LOGFILE.File=${log.dir}/${log.file}
          log4j.appender.LOGFILE.layout=org.apache.log4j.PatternLayout
          log4j.appender.LOGFILE.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
          
          Show
          Arvind Prabhakar added a comment - Suggested configuration: log4j.rootLogger=INFO,LOGFILE log4j.appender.LOGFILE=org.apache.log4j.RollingFileAppender log4j.appender.LOGFILE.MaxFileSize=10MB log4j.appender.LOGFILE.MaxBackupIndex=10 log4j.appender.LOGFILE.File=${log.dir}/${log.file} log4j.appender.LOGFILE.layout=org.apache.log4j.PatternLayout log4j.appender.LOGFILE.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
          Hide
          Inder SIngh added a comment -

          isn't the MaxFileSize a little conservative, considering people would mostly use defaults.

          Show
          Inder SIngh added a comment - isn't the MaxFileSize a little conservative, considering people would mostly use defaults.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4565/
          -----------------------------------------------------------

          Review request for Flume.

          Summary
          -------

          Simple patch to default to a reasonably-configured rolling file appender.

          This addresses bug FLUME-1073.
          https://issues.apache.org/jira/browse/FLUME-1073

          Diffs


          conf/log4j.properties 3dcffd4

          Diff: https://reviews.apache.org/r/4565/diff

          Testing
          -------

          Did some manual testing, looks fine to me.

          Thanks,

          Mike

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4565/ ----------------------------------------------------------- Review request for Flume. Summary ------- Simple patch to default to a reasonably-configured rolling file appender. This addresses bug FLUME-1073 . https://issues.apache.org/jira/browse/FLUME-1073 Diffs conf/log4j.properties 3dcffd4 Diff: https://reviews.apache.org/r/4565/diff Testing ------- Did some manual testing, looks fine to me. Thanks, Mike
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4565/#review6561
          -----------------------------------------------------------

          Ship it!

          +1. Please attach the patch to the Jira.

          nit: we should add the console appender as well since up until now the output was visible on the console only.

          • Arvind

          On 2012-03-30 00:54:39, Mike Percy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4565/

          -----------------------------------------------------------

          (Updated 2012-03-30 00:54:39)

          Review request for Flume.

          Summary

          -------

          Simple patch to default to a reasonably-configured rolling file appender.

          This addresses bug FLUME-1073.

          https://issues.apache.org/jira/browse/FLUME-1073

          Diffs

          -----

          conf/log4j.properties 3dcffd4

          Diff: https://reviews.apache.org/r/4565/diff

          Testing

          -------

          Did some manual testing, looks fine to me.

          Thanks,

          Mike

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4565/#review6561 ----------------------------------------------------------- Ship it! +1. Please attach the patch to the Jira. nit: we should add the console appender as well since up until now the output was visible on the console only. Arvind On 2012-03-30 00:54:39, Mike Percy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4565/ ----------------------------------------------------------- (Updated 2012-03-30 00:54:39) Review request for Flume. Summary ------- Simple patch to default to a reasonably-configured rolling file appender. This addresses bug FLUME-1073 . https://issues.apache.org/jira/browse/FLUME-1073 Diffs ----- conf/log4j.properties 3dcffd4 Diff: https://reviews.apache.org/r/4565/diff Testing ------- Did some manual testing, looks fine to me. Thanks, Mike
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4565/
          -----------------------------------------------------------

          (Updated 2012-03-30 21:38:31.640395)

          Review request for Flume.

          Changes
          -------

          Updated patch so that we can now specify -Dflume.root.logger=DEBUG,console when invoking Flume on the command line, which overrides the log4j.rootLogger. I think that's a reasonable thing to ask of developers. At the same time, "out of the box", Flume has logging that is sane for a production environment.

          Summary
          -------

          Simple patch to default to a reasonably-configured rolling file appender.

          This addresses bug FLUME-1073.
          https://issues.apache.org/jira/browse/FLUME-1073

          Diffs (updated)


          bin/flume-ng a31f171
          conf/log4j.properties 3dcffd4

          Diff: https://reviews.apache.org/r/4565/diff

          Testing
          -------

          Did some manual testing, looks fine to me.

          Thanks,

          Mike

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4565/ ----------------------------------------------------------- (Updated 2012-03-30 21:38:31.640395) Review request for Flume. Changes ------- Updated patch so that we can now specify -Dflume.root.logger=DEBUG,console when invoking Flume on the command line, which overrides the log4j.rootLogger. I think that's a reasonable thing to ask of developers. At the same time, "out of the box", Flume has logging that is sane for a production environment. Summary ------- Simple patch to default to a reasonably-configured rolling file appender. This addresses bug FLUME-1073 . https://issues.apache.org/jira/browse/FLUME-1073 Diffs (updated) bin/flume-ng a31f171 conf/log4j.properties 3dcffd4 Diff: https://reviews.apache.org/r/4565/diff Testing ------- Did some manual testing, looks fine to me. Thanks, Mike
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4565/#review6612
          -----------------------------------------------------------

          Ship it!

          +1.

          Thanks for the patch Mike. Please attach the patch to the Jira.

          • Arvind

          On 2012-03-30 21:38:31, Mike Percy wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4565/

          -----------------------------------------------------------

          (Updated 2012-03-30 21:38:31)

          Review request for Flume.

          Summary

          -------

          Simple patch to default to a reasonably-configured rolling file appender.

          This addresses bug FLUME-1073.

          https://issues.apache.org/jira/browse/FLUME-1073

          Diffs

          -----

          bin/flume-ng a31f171

          conf/log4j.properties 3dcffd4

          Diff: https://reviews.apache.org/r/4565/diff

          Testing

          -------

          Did some manual testing, looks fine to me.

          Thanks,

          Mike

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4565/#review6612 ----------------------------------------------------------- Ship it! +1. Thanks for the patch Mike. Please attach the patch to the Jira. Arvind On 2012-03-30 21:38:31, Mike Percy wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4565/ ----------------------------------------------------------- (Updated 2012-03-30 21:38:31) Review request for Flume. Summary ------- Simple patch to default to a reasonably-configured rolling file appender. This addresses bug FLUME-1073 . https://issues.apache.org/jira/browse/FLUME-1073 Diffs ----- bin/flume-ng a31f171 conf/log4j.properties 3dcffd4 Diff: https://reviews.apache.org/r/4565/diff Testing ------- Did some manual testing, looks fine to me. Thanks, Mike
          Hide
          Arvind Prabhakar added a comment -

          Patch committed. Thanks Mike!

          Show
          Arvind Prabhakar added a comment - Patch committed. Thanks Mike!
          Hide
          Hudson added a comment -

          Integrated in flume-trunk #154 (See https://builds.apache.org/job/flume-trunk/154/)
          FLUME-1073. Default log4j configuration file should have rolling policy.

          (Mike Percy via Arvind Prabhakar) (Revision 1308208)

          Result = SUCCESS
          arvind : http://svn.apache.org/viewvc/?view=rev&rev=1308208
          Files :

          • /incubator/flume/trunk/bin/flume-ng
          • /incubator/flume/trunk/conf/log4j.properties
          Show
          Hudson added a comment - Integrated in flume-trunk #154 (See https://builds.apache.org/job/flume-trunk/154/ ) FLUME-1073 . Default log4j configuration file should have rolling policy. (Mike Percy via Arvind Prabhakar) (Revision 1308208) Result = SUCCESS arvind : http://svn.apache.org/viewvc/?view=rev&rev=1308208 Files : /incubator/flume/trunk/bin/flume-ng /incubator/flume/trunk/conf/log4j.properties

            People

            • Assignee:
              Mike Percy
              Reporter:
              Arvind Prabhakar
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development