[FLUME-3094] when some of file was rolling by log4j, I fond a bug of this tailsource

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.7.0
• Component/s: Sinks+Sources
• Assignee: Unassigned
• Reporter: roger
• Created: 19/May/17 08:39
• Updated: 24/May/17 03:16

Description

Hi,
I fond a bug of this tailsource.
When I use the regular expression to match files under a folder. when some of file was rolling by log4j, this file's start position which is used to record tail position will be setted to 0. And then all of matched files will be readed again.
after checking the code. I find below info:
The process() will update all of inodes info by "existingInodes.addAll(reader.updateTailFiles());"
But the the skipToEnd will be setted to "false" when update this file.
" public List<Long> updateTailFiles() throws IOException

{ return updateTailFiles(false); }

"
when this file was rolled. below code will be executed. this startPos will be setted to 0. It will be readed again.
if (tf == null || !tf.getPath().equals(f.getAbsolutePath()))

{ long startPos = skipToEnd ? f.length() : 0; tf = openFile(f, headers, inode, startPos); }

Does anyone occurred same problem or is there any setting I missed?