Flume
  1. Flume
  2. FLUME-1314

File channel log file can grow beyond max size which causes startup failure

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.2.0
    • Component/s: None
    • Labels:
      None

      Description

      Starting the channel fails with the following exception:

      2012-06-24 03:05:32,517 INFO file.LogFile: fast-forward to checkpoint position: 2146687031
      2012-06-24 03:05:32,518 ERROR file.Log: Failed to initialize Log
      java.lang.IllegalStateException: 2146687031
              at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
              at org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:335)
              at org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:88)
              at org.apache.flume.channel.file.Log.replay(Log.java:229)
              at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:202)
              at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:228)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
              at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
              at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
              at java.lang.Thread.run(Thread.java:662)
      2012-06-24 03:05:32,521 ERROR file.FileChannel: Failed to start the file channel
      
      1. FLUME-1314-2.patch
        7 kB
        Arvind Prabhakar
      2. FLUME-1314-1.patch
        2 kB
        Arvind Prabhakar

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          1d 8h 6m 1 Arvind Prabhakar 25/Jun/12 20:00
          In Progress In Progress Patch Available Patch Available
          27s 1 Arvind Prabhakar 25/Jun/12 20:00
          Patch Available Patch Available Resolved Resolved
          12h 1 Mike Percy 26/Jun/12 08:01
          Hide
          Hudson added a comment -

          Integrated in flume-trunk #257 (See https://builds.apache.org/job/flume-trunk/257/)
          FLUME-1314. Log file size violation prevents FileChannel startup.

          (Arvind Prabhakar via Mike Percy) (Revision 1353836)

          Result = UNSTABLE
          mpercy : http://svn.apache.org/viewvc/?view=rev&rev=1353836
          Files :

          • /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannelConfiguration.java
          • /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java
          • /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestLog.java
          Show
          Hudson added a comment - Integrated in flume-trunk #257 (See https://builds.apache.org/job/flume-trunk/257/ ) FLUME-1314 . Log file size violation prevents FileChannel startup. (Arvind Prabhakar via Mike Percy) (Revision 1353836) Result = UNSTABLE mpercy : http://svn.apache.org/viewvc/?view=rev&rev=1353836 Files : /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannelConfiguration.java /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java /incubator/flume/trunk/flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestLog.java
          Mike Percy made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Mike Percy added a comment -

          Patch committed. Thanks Arvind!

          Show
          Mike Percy added a comment - Patch committed. Thanks Arvind!
          Arvind Prabhakar made changes -
          Attachment FLUME-1314-2.patch [ 12533397 ]
          Arvind Prabhakar made changes -
          Remote Link This issue links to "Review Request (Web Link)" [ 10212 ]
          Arvind Prabhakar made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Fix Version/s v1.2.0 [ 12320243 ]
          Hide
          Arvind Prabhakar added a comment -

          Attached the patch.

          Show
          Arvind Prabhakar added a comment - Attached the patch.
          Arvind Prabhakar made changes -
          Attachment FLUME-1314-1.patch [ 12533355 ]
          Arvind Prabhakar made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Will McQueen made changes -
          Field Original Value New Value
          Summary File channel log file can grow beyond log size which causes startup failure File channel log file can grow beyond max size which causes startup failure
          Arvind Prabhakar created issue -

            People

            • Assignee:
              Arvind Prabhakar
              Reporter:
              Arvind Prabhakar
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development