Details
Major Description
Using this config:
logs-2 default-flow collectorSource(9109) collectorSink("file:///data/log/flumed/%
/dt=%Y-%m-%d", "%k-json", 900000)
agent1 default-flow rpcSource(9108) agentBESink("logs-2", 9109)
(assuming a max body size of 30kb)
If an event is passed through agent1 => logs-2 where the body is larger than 30kb then during file roll logs-2 will SILENTLY fail with repeated exception messages in the log files (DEBUG severity)
[see attached log snippet]
The only log entry of higher importance than INFO is this one:
WARN com.cloudera.flume.handlers.rolling.RollSink: TriggerThread interrupted
which is totally useless for debugging
Other information / speculation:
1) The same thing happens on agent1 during the agentSink if you surround the agentSink with collector(15000){}. When it rolls the connection the same exception and behavior occur. Basically it happens whenever there is rolling.
2) I'm not sure if any event larger than 30kb triggers this, or whether it has to receive the event at a particular time relative to the file roll.
3) I have no idea why this happens at roll-time and not during regular event collection.
4) I don't know whether this is directly related to the use of the rpcSink. I know the internal communication mechanisms share code with the sink so perhaps the regular event checks do not happen?
5) using the flume shell, running getnodestatus will tell you that all nodes are active, despite this problem.