[FLUME-248] TailSource not closing files

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.9.1, 0.9.1u1
• Fix Version/s: 0.9.2
• Component/s: Node, Sinks+Sources
• Assignee: Jonathan Hsieh
• Reporter: Disabled imported user
• Created: 28/Sep/10 14:33
• Updated: 13/Dec/10 14:37
• Resolved: 04/Nov/10 21:01

Description

After logs rotated last night, the agent is still holding on to the log file it was tailing:

java 14686 flume 41r REG 8,17 5989827 34996343 /var/log/httpd/logfile.log.1 (deleted)
java 14686 flume 44r REG 8,17 5989827 34996343 /var/log/httpd/logfile.log.1 (deleted)

The logs get rotated, then gzipped and the originals are deleted. It looks like the same issue may happen in other portions of the code; Bruce mentioned the RandomAccessFile objects in the reporter don't close files properly either.

Agent configuration:
agent1: tail( "/var/log/httpd/logfile.log", true ) | agentE2EChain( "collector1", "collector2", "collector3" );

Issue Links

• is related to: FLUME-252 Update Tail to get rid of races and truncation problems.
• relates to: FLUME-234 Flume agent node not always closing deleted files
• relates to: FLUME-235 Flume agent holds on to deleted file handles