[FLUME-2528] MalformedInputException when parsing new log file

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.5.0.1
• Labels: MalformedInputException

Environment

Red Hat Enterprise Linux Server release 6.4
java version "1.6.0_45"

• Assignee: Unassigned
• Reporter: kext
• Created: 29/Oct/14 09:59
• Updated: 29/Oct/14 13:15
• Resolved: 29/Oct/14 13:15

Description

Below is my processing steps:
1. put the log file into /home/dev/logs(configure log path)
2. start flume, the result is normal.

but, if i reverse the steps to be liked below:
1. start flume
2. put the same log file into the configure log path

then get below exception.

Exception details:
2014-10-29 17:26:47,125 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source src:

{ spoolDir: /home/dev/logs/ }

: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.
java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)
at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:134)
at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:72)
at org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:91)
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:238)
at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:227)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)

I really don't know the reason because the testing log files are same, and i found that #2052# is a little similar with my issue, but i don't think that they are same, may i know if anyone can give me some help? thanks.