Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
1.9.0
-
None
-
None
-
All Flume versions
Description
Hello ,
I don't know if anybody faced a bug with TaildirSource backoffWithoutNL。
when souce log files, sometimes , i got lines break into 2 Incomplete lines, and
the backoffWithoutNL did not work,because the file is new arrival and markerd as idel wrongly。
//代码占位符 public TailFile(File file, Map<String, String> headers, long inode, long pos) throws IOException { this.raf = new RandomAccessFile(file, "r"); if (pos > 0) { raf.seek(pos); lineReadPos = pos; } this.path = file.getAbsolutePath(); this.inode = inode; this.pos = pos; this.lastUpdated = 0L; // here, the TailFile Construtor init lastUpdated to 0L this.needTail = true; this.headers = headers; this.oldBuffer = new byte[0]; this.bufferPos = NEED_READING; }
the TailFile Construtor init lastUpdated to 0L,and lastUpdated only change when commit() and close()。
//代码占位符 private class idleFileCheckerRunnable implements Runnable { @Override public void run() { try { long now = System.currentTimeMillis(); for (TailFile tf : reader.getTailFiles().values()) { // here, the new file can be mark as idel wrongly if (tf.getLastUpdated() + idleTimeout < now && tf.getRaf() != null) { idleInodes.add(tf.getInode()); } } } catch (Throwable t) { logger.error("Uncaught exception in IdleFileChecker thread", t); } } }
i change the TailFile Construtor,
this.lastUpdated = System.currentTimeMillis();
so on , there's no break line any more