[FLUME-1400] Understand and document behavior of exec source when tailing rolling logs

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Assignee: NO NAME
• Reporter: NO NAME
• Created: 28/Jul/12 00:19
• Updated: 01/Aug/12 01:04
• Resolved: 01/Aug/12 00:55

Description

There has been some discussion recently around the behavior of exec("tail -f") when tailing logs, if the log files are themselves being rolled and renamed by a writer process.

The primary concern is whether there is a race condition or correctness problem related to tailing a log which is renamed.

I am fairly certain that the tail binary, when passed -f, will deal with all of the ugliness of recognizing when a file is rolled and open the new "current" file as expected. However, we should both verify and document that behavior.

Flume OG, which implemented its own tailing functionality, dealt with this ugliness by introducing increasing break-fix complexity to the tail source. I am hoping that by delegating to the "tail" binary we have avoided having to do this (though at the cost of OS portability). Nonetheless, I should confirm this.