Flume
  1. Flume
  2. FLUME-1108

FILE_ROLL sink doesn't accept value 0 for unlimited wait time before rolling file

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.2.0
    • Fix Version/s: v1.2.0
    • Component/s: Sinks+Sources
    • Labels:
      None
    • Environment:

      [CentOS 5.7 64-bit]

      Description

      Using FILE_ROLL sink and the following props for it:
      sink.directory=/home/will/logs
      sink.rollInterval=0

      ...the logical expectation should should be that the file never gets rolled – all output should go to single file.

      2012-04-06 10:50:33,272 ERROR lifecycle.LifecycleSupervisor: Unable to start SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@d6089a5 counterGroup:{ name:null counters:{} } } - Exception follows.
      java.lang.IllegalArgumentException
      at java.util.concurrent.ScheduledThreadPoolExecutor.scheduleAtFixedRate(ScheduledThreadPoolExecutor.java:420)
      at org.apache.flume.sink.RollingFileSink.start(RollingFileSink.java:104)
      at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:43)
      at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
      at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:221)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:662)

      1. FLUME-1108-1.patch
        5 kB
        Hari Shreedharan

        Activity

        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4672/
        -----------------------------------------------------------

        Review request for Flume.

        Summary
        -------

        If rollInterval is set to 0, RollingFileSink does not roll file.

        This addresses bug FLUME-1108.
        https://issues.apache.org/jira/browse/FLUME-1108

        Diffs


        flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 7f1d3b3
        flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java 7e26e2a

        Diff: https://reviews.apache.org/r/4672/diff

        Testing
        -------

        Added unit test.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4672/ ----------------------------------------------------------- Review request for Flume. Summary ------- If rollInterval is set to 0, RollingFileSink does not roll file. This addresses bug FLUME-1108 . https://issues.apache.org/jira/browse/FLUME-1108 Diffs flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 7f1d3b3 flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java 7e26e2a Diff: https://reviews.apache.org/r/4672/diff Testing ------- Added unit test. Thanks, Hari
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4672/#review6753
        -----------------------------------------------------------

        Ship it!

        hmm .. disable rolling for for a component called 'rolling file sink', that's like ordering Ice tea with no ice This probably not be useful in a real production cases, but certainly very handy for testing/validating.

        Changes look fine to me.

        • Prasad

        On 2012-04-06 19:29:44, Hari Shreedharan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4672/

        -----------------------------------------------------------

        (Updated 2012-04-06 19:29:44)

        Review request for Flume.

        Summary

        -------

        If rollInterval is set to 0, RollingFileSink does not roll file.

        This addresses bug FLUME-1108.

        https://issues.apache.org/jira/browse/FLUME-1108

        Diffs

        -----

        flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 7f1d3b3

        flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java 7e26e2a

        Diff: https://reviews.apache.org/r/4672/diff

        Testing

        -------

        Added unit test.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4672/#review6753 ----------------------------------------------------------- Ship it! hmm .. disable rolling for for a component called 'rolling file sink', that's like ordering Ice tea with no ice This probably not be useful in a real production cases, but certainly very handy for testing/validating. Changes look fine to me. Prasad On 2012-04-06 19:29:44, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4672/ ----------------------------------------------------------- (Updated 2012-04-06 19:29:44) Review request for Flume. Summary ------- If rollInterval is set to 0, RollingFileSink does not roll file. This addresses bug FLUME-1108 . https://issues.apache.org/jira/browse/FLUME-1108 Diffs ----- flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 7f1d3b3 flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java 7e26e2a Diff: https://reviews.apache.org/r/4672/diff Testing ------- Added unit test. Thanks, Hari
        Hide
        Prasad Mujumdar added a comment -

        @Hari, please attach the patch on Jira. I will commit it to trunk.

        Show
        Prasad Mujumdar added a comment - @Hari, please attach the patch on Jira. I will commit it to trunk.
        Hide
        Hari Shreedharan added a comment -

        Hi Prasad - I had already attached it to the jira. Thanks!

        Show
        Hari Shreedharan added a comment - Hi Prasad - I had already attached it to the jira. Thanks!
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4672/#review6770
        -----------------------------------------------------------

        Ship it!

        LGTM!
        Yes, I think it would be useful especially in testing scenario. This can be considered the degenerate case, and doesn't affect existing functionality. It's consistent with semantics for other sources/sinks where "0" value is specified. Saves user from needing to enter a high value for rolloverInterval when running multi-day endurance tests.

        • Will

        On 2012-04-06 19:29:44, Hari Shreedharan wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4672/

        -----------------------------------------------------------

        (Updated 2012-04-06 19:29:44)

        Review request for Flume.

        Summary

        -------

        If rollInterval is set to 0, RollingFileSink does not roll file.

        This addresses bug FLUME-1108.

        https://issues.apache.org/jira/browse/FLUME-1108

        Diffs

        -----

        flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 7f1d3b3

        flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java 7e26e2a

        Diff: https://reviews.apache.org/r/4672/diff

        Testing

        -------

        Added unit test.

        Thanks,

        Hari

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4672/#review6770 ----------------------------------------------------------- Ship it! LGTM! Yes, I think it would be useful especially in testing scenario. This can be considered the degenerate case, and doesn't affect existing functionality. It's consistent with semantics for other sources/sinks where "0" value is specified. Saves user from needing to enter a high value for rolloverInterval when running multi-day endurance tests. Will On 2012-04-06 19:29:44, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4672/ ----------------------------------------------------------- (Updated 2012-04-06 19:29:44) Review request for Flume. Summary ------- If rollInterval is set to 0, RollingFileSink does not roll file. This addresses bug FLUME-1108 . https://issues.apache.org/jira/browse/FLUME-1108 Diffs ----- flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java 7f1d3b3 flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java 7e26e2a Diff: https://reviews.apache.org/r/4672/diff Testing ------- Added unit test. Thanks, Hari
        Hide
        Prasad Mujumdar added a comment -

        Patch committed to trunk.
        Thanks Hari !

        Show
        Prasad Mujumdar added a comment - Patch committed to trunk. Thanks Hari !
        Hide
        Hudson added a comment -

        Integrated in flume-trunk #164 (See https://builds.apache.org/job/flume-trunk/164/)
        FLUME-1108: FILE_ROLL sink doesn't accept value 0 for unlimited wait time before rolling file
        (Hari Shreedharan via Prasad Mujumdar) (Revision 1311144)

        Result = SUCCESS
        prasadm : http://svn.apache.org/viewvc/?view=rev&rev=1311144
        Files :

        • /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java
        • /incubator/flume/trunk/flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java
        Show
        Hudson added a comment - Integrated in flume-trunk #164 (See https://builds.apache.org/job/flume-trunk/164/ ) FLUME-1108 : FILE_ROLL sink doesn't accept value 0 for unlimited wait time before rolling file (Hari Shreedharan via Prasad Mujumdar) (Revision 1311144) Result = SUCCESS prasadm : http://svn.apache.org/viewvc/?view=rev&rev=1311144 Files : /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java /incubator/flume/trunk/flume-ng-core/src/test/java/org/apache/flume/sink/TestRollingFileSink.java

          People

          • Assignee:
            Hari Shreedharan
            Reporter:
            Will McQueen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development