-
Type:
Bug
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 1.7.0
-
Fix Version/s: 1.9.0
-
Component/s: Sinks+Sources
-
Labels:
If there are many files in the path that need to be tail, and there is a file written by high frequency (for example, there are file a, file b and file c in the path, file a is written at high frequency), taildir can read the batchSize size event from file a everytime, then taildir will only read data from file a, other files will not to be read, because in TaildirSource.tailFileProcess will into an endless loop.
code:
TaildirSource.java
private void tailFileProcess(TailFile tf, boolean backoffWithoutNL) throws IOException, InterruptedException { while (true) { // if events.size >= batchSize will not break while, // then into endless loop to only read tf if (events.size() < batchSize) { break; } } }
- relates to
-
FLUME-3294 Fix polling logic in TaildirSource
-
- Resolved
-
- links to