Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2376

File Channel corrupted - java.io.IOException: Unable to read next Transaction from log file

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.5.0
    • Fix Version/s: None
    • Component/s: File Channel
    • Labels:
      None
    • Environment:

      windows

      Description

      I am using filechannel with dual checkpoint enabled.
      getting below error when I restart my agent.
      earlier process was abruptly killed. filechannel can not be recovered from state.
      Actually i am using version 1.4.0 with patch for https://issues.apache.org/jira/browse/FLUME-2307

      is there way to recover filechannel data.

      --------- stack trace -----------
      ERROR: Failed to initialize Log on [channel=c2]
      java.io.IOException: Unable to read next Transaction from log file /data1/collection/data/c2/flume-data/log-198 at offset 69616
      at org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:629) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.ReplayHandler.next(ReplayHandler.java:383) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:280) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.Log.doReplay(Log.java:480) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.Log.replay(Log.java:407) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:273) [my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251) [my-agent-1.0-SNAPSHOT.jar:na]
      at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [na:1.7.0_45]
      at java.util.concurrent.FutureTask.runAndReset(Unknown Source) [na:1.7.0_45]
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source) [na:1.7.0_45]
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [na:1.7.0_45]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_45]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_45]
      at java.lang.Thread.run(Unknown Source) [na:1.7.0_45]
      Caused by: com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: type, transactionID, writeOrderID
      at com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(UninitializedMessageException.java:81) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.proto.ProtosFactory$TransactionEventHeader$Builder.buildParsed(ProtosFactory.java:2912) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.proto.ProtosFactory$TransactionEventHeader$Builder.access$5200(ProtosFactory.java:2846) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.proto.ProtosFactory$TransactionEventHeader.parseDelimitedFrom(ProtosFactory.java:2804) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.TransactionEventRecord.fromByteArray(TransactionEventRecord.java:196) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.LogFileV3$SequentialReader.doNext(LogFileV3.java:359) ~[my-agent-1.0-SNAPSHOT.jar:na]
      at org.apache.flume.channel.file.LogFile$SequentialReader.next(LogFile.java:625) ~[my-agent-1.0-SNAPSHOT.jar:na]
      ... 13 common frames omitted
      Apr 28, 2014 11:43:02 PM org.apache.flume.channel.file.FileChannel start():284
      ERROR: Failed to start the file channel [channel=c2]

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              arajput Arun
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: