[FLUME-2430] Losing events with multiple agents flow

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.5.0, 1.5.0.1

Environment

Win7 64bit, Cygwin

• Assignee: Unassigned
• Reporter: Vitaly
• Created: 22/Jul/14 15:02
• Updated: 21/Aug/14 10:23

Description

Not all events are delivered and persisted to file when using multi agent flow:

agent1.sources = avroSource1
agent1.sinks = avroSync
agent1.channels = memoryChannel

agent1.sources.avroSource1.type = avro
agent1.sources.avroSource1.bind = localhost
agent1.sources.avroSource1.port = 41414

agent1.sinks.avroSync.type = avro
agent1.sinks.avroSync.hostname = localhost
agent1.sinks.avroSync.port = 4444

agent1.channels.memoryChannel.type = memory
agent1.channels.memoryChannel.capacity = 10000
agent1.channels.memoryChannel.transactionCapacity = 1000

agent1.sources.avroSource1.channels = memoryChannel
agent1.sinks.avroSync.channel = memoryChannel

agent2.sources = avroSource
agent2.sinks = fileRollSink
agent2.channels = memoryChannel

agent2.sources.avroSource.type = avro
agent2.sources.avroSource.bind = localhost
agent2.sources.avroSource.port = 4444

agent2.sinks.fileRollSink.type = file_roll
agent2.sinks.fileRollSink.sink.directory = C:\\Users\\flume
flumeFileAgent
agent2.sinks.fileRollSink.sink.rollInterval=0

agent2.channels.memoryChannel.type = memory
agent2.channels.memoryChannel.capacity = 10000
agent2.channels.memoryChannel.transactionCapacity = 1000

agent2.sources.avroSource.channels = memoryChannel
agent2.sinks.fileRollSink.channel = memoryChannel

For example I am passing 1000 events to the first "agent1" but "agent2" doesn't receive from "agent1" all of them, only 800 or 700 are stored in file. No exceptions are thrown by flume agents.